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The purpose of the demonstration programs in 
intensive reading and mathematics instruction for low-achieving 
students was essentially to enable school districts to establish and 
operate exemplary and innovative projects to improve the competence 
of junior high school students in reading and mathematics. Projects 
were limited to students in grades seven, eight, and nine attending 
schools located in low-income areas. Participants were educationally 
disadvantaged students who would otherwise find difficulty i.n 
achieving success in high school. Project proposals were required to 
contain statements of specific goals with respect to student 
achievement and to show a level of cost effectiveness that would make 
it possible for other, similar school districts in California to 
adapt the projects to their needs. Projects that proved least cost 
effective were to be terminated. Seventeen projects were approved and 
1970-71 was their second year of operation. Projects were continually 
revie%«ed by the Division of compensatory Education regarding their 
effectiveness in improving the achievement level of students in 
reading and mathematics. Through reports, questionnaires, 
observations, and interviews, projects were evaluated on the basis of 
several criteria involving program development, student achievement, 
and cost analysis. (Author/JN) 
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Overview of the Demonstration Project 






The California State Departnnent of Education is in the proceaa of creating 
more accountable educational programs throughout the state. The project 
described in this report --a report required by Education Code Section 6497 -- 
is a step toward a model of accountability. 

With the passage of Assembly Bill 938, the 1969 Legislature authorized the 
Division of Compensatory Education to establish demonstration programs in 
intensive reading and mathematics instruction that would improve the academic 
achievement of low^achieving students and be cost effective. Demonstration 
programs were established in 17 of the poorest schools in California with 
respect to their students' socioeconomic status and academic achievement. 

In terms of educational failure and lack of hope, these were the schools that 
had the greatest need for such programs. 

The students in the most cost-effective demonstration programs have shown 
a rate of achievement seen in few compensatory education programs anywhere 
in the nation. These programs were designed with the objective of assisting 
students to attain a more normal range and distribution of achievement. This 
objective was achieved in almost all cases. The few programs that were not 
successful were terminated. 

The programs are unique in many ways. They are planned and developed 
by the staff of each participating school. The principal, the project director, 
and a carefully selected staff of teachers, aides, and volunteers make decisions 
at the school level. Students with y;aryinp, abilities are grouped together 
heterogeneously. They are usually taught individually or in groups of two or 
three at least part of each day by teachers and assistants. In many programs, 
each student spends some time each week in a learning laboratory where he 
works with highly trained teachers and assistants in specific learning activities 
activities designed especially for him. 

Curriculum is developed and prescribed as needed and is based upon a careful 
diagnosis of learning disabilities. Success is the key to the program. The 
students are made aware of their successes rather than criticized for their 
failures. Students feel the high expectations and in turn are motivated to learn. 

The program attempts to remove many obstacles that hinder learning; for 
example, sections of the Education Code may be waived if their provisions seem 
to interfere with an innovative program. 

Another unique feature of the programs is that, unlike most other state or 
federal projects, those projects that are considered to be the least cost effective 
.are terminated. Funds from, terminated projects are being used to replicate 
cost-effective programs in more schools within districts that have already 
conducted successful projects. 
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The demonstration programs began at the seventh grade level in 1969-70, 
followed the participating students into grade eight in 1970-71, and is serving 
them in grade nine in 1971-72. All eligible students at the apprbpnate grade 
levels in each project school were served. As the state-funded programs 
moved from one grade to the next, the school districts established similar 
programs for incoming students in the grade no longer being served by the 
original programs. Because of the success of their demonstration programs, 
several districts have begun similar programs in additional junior high schools 
within their districts, generally with tremendous impact on traditional instruc- 
tional programs in reading and mathematics. 



Implementation of the Programs 

The purpose of the demonstration programs in intensive reading and 
mathematics instruction for low-achieving students was essentially to enable 
school districts to establish and operate exemplary and innovative projects 
to improve the competence of junior high school students in reading and mathe- 
matics. Seventeen projects were approved, and 1970-71 was their second 
year of operation. 

Projects were limited to students in grades seven, eight, and nine attending 
schools located in low-income areas. Participants were educationally disad- 
vantaged students who would otherwise find difficulty in achieving success/in 
high school. 

Project proposals were required to contain statements of specific goals 
with respect to student achievement and to show a level of cost effectiveness 
that would make it possible for other, similar school districts throughout 
Califoraia to adapt the projects to their needs. Projects that proved least 
cost effective were to be terminated. 

The most effective demonstration programs in reading were those in Colton, 

Los Angeles (Edison), Los Angeles (Pacoima), Riverside, and Santa Barbara. 

The most effective mathematics programs were in Colton, Los Angeles (Pacoima), 
Pasadena, and Riverside. 

Effective demonstration programs in reading included those in El Monte, 
Fresno, Montebello, Oakland, San Diego, San Francisco, and San Jose. 

Effective mathematics programs were implemented in Fresno, Long Beach, 

Los Angeles (Edison), Montebello, Oakland, and San Jose. 

The least effective demonstration programs included the projects in reading 
and mathematics at Los Angeles (Belvedere) and Stockton, the San Francisco 
mathematics program, and the Long Beach reading program. 
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Recommendations Regarding the Programs 



Considering the purpose of the programs as defined by Assembly Bill 938, 
evaluation criteria, and evaluation results, the following recommendations 
are offered regarding the demonstration programs in reading and mathematics: 



1. Demonstration programs in the following districts should be retained and 
extended; they should be commended as most effective; and efforts should 
be directed toward greater demonstration and dissemination of information 
regarding their most innovative and exemplary cost-effective elements: 



Reading programs 



Mathematics programs 



Colton Joint Unified *' 

Los Angeles Unified (Pacoima) 
Riverside Unified 
Santa Barbara City Elementary 
Los Angeles Unified (Edison) 



C9lton Joint Unified 
Los Angeles Unified (Pacoima) 
Pasadena Unified 
Riverside Unified 



■"s' 
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The demonstration program in the Montebello Unified School District should 
be commended for its excellence, although the district did not apply for 
renewal of the project for 1971-72. The division of Compensatory Education 
should continue further dissemination of information about the program and 
this district's outstanding replication of exemplary elements in the seventh 
and eighth grades. 



I . 



3. Effective demonstration programs in the following districts should' 

retained and strengthened, and efforts should be directed toward modifying 
these programs to improve their program effectiveness and cost effective- 
ness and increase their achievement gains: 






Reading programs 



Mathematics programs 



San Francisco Unified 
El Monte Elementary 
Oakland City Unified 
San Jose Unified 
San Diego City Unified 
Fresno City Unified 



Long Beach Unified 

Los Angeles Unified (Edison) 

Oakland City Unified 

San Jose Unified 

Fresno City Unified 



4. Demonstration programs in the following districts should be terminated 
as least effective for the reasons indicated: 



a. 



Los Angeles Unified (Belvedere) (reading and mathematics). The Belvedere 
project rated low on organization and administration, program development, 
and fiscal management and lowest of all projects according to an overall 
evaluation questionnaire (rank: seventeenth). The project also rated low 
in project observation data, low on a visitation inventory, and low on overall 
program effectiveness (rardc: fourteenth). The project ranked 12.5 out of 
15 projects in reading achievement and eleventh out of 13 projects in mathe- 
matics achievement gains. The project ranked in the lowest quarter in 
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reading achievement, mathematics achievement, and program effective- 
ness. In overall rank order (weighted 40 percent achievement, 40 percent 
costs, 20 percent program). Belvedere ranked fifteenth out of 15 reading 
projects and 10.5 out of 13 mathematics projects. 

b. Stockton City Unified (reading and mathematics). The Stockton project 

. tied another district for lowest place on the program visitation inventory 
(rank: 15.5 out of 18). The project rated low in project observation 
data and in overall program effectiveness (rank order: 11.5). Baseline 
data for 1969-70 revealed low ratings in the evaluation report, in reading 

' ' achievement gains, and^c^sts per student. The project rated lowest 
among 15 reading programs in reading achievement gains (0.8 year's 
growth per year) and rated s'econd lowest in mathematics gains (rank; 
eleventh out of 13). The project showed high operating costs (rank: 
tenth) and, in cost effectivenesc^ ranked ninth out of 15 reading projects 
and ninth out of 13 mathematics projects. It ranked in the lowest quarter 
on reading and mathematics gains and below the median on operating 
costs and program effectiveness. In overall rank order (weighted 40 percent 
achievement, 40 percent costs, 20 percent program), Stockton ranked 
twelfth out of 15 reading projects and 10.5 out of 13 mathematics projects. 

c. San Francisco Unified (mathematics). The San Francisco mathematics 
program' ''rated low on the program evaluation questionnaire (rank order: 
twelfth out of 17 programs). Tt rated lowest of all programs in achievement 
gains (rank: thirteenth) -- achievement gains reported were equal to four 
months per year of instruction. High expenditures per student were 
reported for operating costs (second highest, ranking twelfth out of 13 
mathematics programs) and for total expenditures per student. The 
program was the least cost effective of all 13 mathematics programs 

in mathematics, operating, and total costs and in overall cost effectiveness. 
The program ranked in the lowest quarter in mathematics achievement and 
operating costs. In overall rank order (weighted 40 percent achievement, 

40 percent costs, 20 percent program), the San Francisco mathematics 
program placed last, ranking thirteenth out of 13 projects. 
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Long Beach Unified (reading). The Long Beach reading program rated 
high in program effectiveness, but relatively low in reading achievement 
(rank: 10.5 out of 15). The program ranked highest of 15 programs in 
reading costs, second highest in research and development costs, second 
highest in operating costs, and third highest in total expenditures per 
student. The program was the least cost effective of all 15 reading, programs 
in reading, operating, and total costs and in overall cost effectiveness. 

The program ranked in the lowest quarter in operating costs per student. 

In overall rank order (weighted 40 percent achievement, 40 percent costs, 

20 percent program), the Long Beach reading program ranked twelfth out 
of 15 projects. r 
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Evaluation of Demonstration Projects 



Projects were continually reviewed by the Division of Compensatory n 

Education regarding their effectiveness in improving the achievement level 
of students in reading and mathematics. Through reports, questionnaires, 
observations, and interviews, projects were evaluated on the basis of 
several criteria involving program development, student achievement, and 
cost analysis. 

Program development criteria included the extent to which the projects 
adhered to the intent of the legislation and State Department of Education 
guidelines on eligibility, selection of participants, waivers, project orgaiii* 
zation and administration, program content, demonstration concepts, staff 
development, and dissemination of information. 

Student achievement criteria included the extent to which the projects 
met objectives, measured results, and demonstrated effectiveness in 
improving student achievement levels. 

Cost analysis criteria included the extent to which the projects accounted 
for component costs and demonstrated cost effectiveness. 

Program Development Criteria 

}/ 

Three separate ratings on program development criteria were combined 
to give a composite rank order of projects on program effectiveness. (See 
Table !.♦) 

First, projects were rated on a 14- point program visitation inventory, 
assessing (1) cost-effective elements implemented in grade seven; (2) cost- 
effective elements implemented in grade eight; (3) proportion of students 
served by the program; (4) demonstration opportunities; (5) performance 
objectives; (6) grouping practices; (7) planning; (8) inservice education; (9) 
dissemination of information; (10) individualization of instruction; (11) use 
of material and personnel; (12) innovative and exemplary program content; 
and (13) staff support. Projects were compared and ranked according to the 
ratings they received on this inventory. Table 1 reveals that eight projects 
tied for highest place on the program visitation inventory; Santa Barbara, 

El Monte, Colton, Riverside, Long Beach (mathematics). Long Beach (reading), 
San Francisco, and San Diego. Projects rated lowest were Stockton and 
Pasadena. Fresno, Los Angeles (Pacoima), Los Angeles (Belvedere), 

Los Angeles (Edison), and San Jose were also rated low. 



*A11 tables referred to in this publication will be found in Appendix A. 
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Projects were rated on a 78* item evaluation questionnaire, assessing the 
extent to which they adhered to guidelines on (1) eligibility; (2) selection of 
participants: (3) waivers; (4) program organization; (5) project administration; 

(6) program content; (7) demonstration activities; (8) staff development: (9) 
dissemination of information; (10) component costs; (11) evaluation procedures; 
and (12) research design. Projects were compared and ranked according to 
point scores on this questionnaire. As indicated in Table 1, Santa Barbara, 

El Monte, Colton, and Long Beach (mathematics) ranked highest on question- 
naire ratings: Los Angeles (Belvedere), Fresno, Los Angeles (Pacoima), and 
Los Angeles (Edison) were rated lowest. 

Projects were also rated on the. basis of subjective observation notes on 
project operations, including interview data compiled during review team 
visits to project schools in the spring of 1971. Projects were compared and 
ranked according to observer ratings based on the general impressions • 
obtained during these visits. Table 1 shows that highest observation ratings 
were given to the projects in Riverside, Santa Barbara, Montebello, and 
El Monte: lowest ratings were assigned to Pasadena, Fresno, Los Angeles 
(Pacoima), and Los Angeles (Belvedere). 

1 / 

These three separate ratings on program development were combined into 
one program effectiveness rating by determining each project's median rank 
order. Projects were then ranked according to this single overall rating. 

Table 1 summarizes these ratings, which placed the following projects highest 
according to the overall rating on program effectiveness; Santa Barbara, 

El Monte. Colton, Riverside, and Long Beach (mathematics). Projects rated 
lowest in program effectiveness were Fresno, Pasadena. Los Angeles (Pacoima), 
and Los Angeles (Belvedere). 

(■' ■ n 

On the informal questionnaire based on AB 938 guidelines, 38 items dis- 
tinguished those projects rated highest from those projects rated lowest. The 
pattern of responses to those questions gave a rough composite profile of the 
factors that tended to differentiate the most effective from the least effective 
demonstration programs. 



Profile of the Most Effective Projects 

In the most effective projects, the programs were being operated in one 
school in the district. The educational needs of the students and the assessment 
of those needs were described in detail. Project data showed very clearly how 
the program met the students' needs. It was evident that the project emphasized 
creativity. The project proposal was excellent and clearly explained the use of 
existing facilities. Over 75 percent of the students in the appropriate grade were 
included in the program. The most general grouping practice used was that of 
random order or heterogeneous groups. Instruction was mostly individualized. 
The project director was located at the school. Most of the project planning was 
done by the whole program staff working with resource personnel. Planning 
included inservice training, a cooperative effort with a business or college, and 
collection of curricular materials. 






The most effective projects were clearly innovative and exemplary. Their 
methods, techniques, and procedures were described in detail, and were clearly 
related to the program objectives. Evidence was usually given of the competence 
of any cooperating agency involved. The impact of the demonstration aspects 
of the programs was such that they were seen as exemplary by other districts, 
and opportunities for observation were well planned and built in. Inservice 
education programs were intensive and effective, continuous, and mandatory. 

The inservice programs were clearly related to the objectives of the projects, 
with schedules and calendars described in the proposals. Each staff was con- 
sidered outstanding, and each program was innovative and unique. Provisions 
were made for dissemination of information through publications, observations, 
and other methods. 

/. V-., 

It was clearly demonstrated at the close of the project that the successful 
districts could maintain their programs, and strong intent was shown to do so. 

The most effective projects showed a level of cost effectiveness that made 
it possible for other, similar school districts to adapt these projects to their 
needs. The average cost per student in the successful projects during the first 
year was from $250 to $500, and the average increase in achievement was from 
11 to 15 months. Therefore, the calculated cost per student per month of growth 
was less than $50. The relationships between program, evaluation, project 
objectives, and expenditures were clearly indicated. The programs were very 
effective in improving the achievement level of students, with a degree of cost 
effectiveness best described as excellent. 



Profile of the Least Effective Projects 

In the least effective projects, the programs were being operated in more 
than one school in the district. The educational needs of the students and the 
assessment of those needs were poorly described. Project data showed fairly 
well how the program met these student needs. It was not evident that the 
unsuccessful projects emphasized creativity. The project proposals contained 
poor explanations of the use of existing facUities. From 25 to 75 percent of 
the students in the appropriate grade were included in the programs. The most 
general grouping practice used was that of homogeneous groups. Instruction 
was only slightly individualized. The project directors were not located at the 
participating schools. Most of the project planning was done by administrators 
not included in the program staff. Planning often omitted inservice training, a 
cooperative effort with a business or college, and collection of curricular 
materials. 

The least effective projects were possibly innovative and exemplary. 

However, their methods, techniques, and procedures were poorly described 
and were only vaguely related to the program objectives. Evidence of the 
competence of any cooperating agency involved was usually lacking. The impact 
of the demonstration aspects of the program was such that the participating 
schools were visited by personnel from other schools in their respective districts 
but not by persons from other districts. Opportunities for observation were very 
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poor. The inservice education programs in the least effective projects were 
fair but perfunctory and neither continuous nor mandatory. The inservice 
programs were somewhat related to the objectives of the projects, but no 
schedules or calendars were described in the proposals. The staffs of the 
unsuccessful projects were considered fair or good. There was scant pro> 
vision for the dissemination of infori^tion through, publications, observations, , 
or other methods. 

In the least effective projects, comparison groups either were not used or 
were not described. The least effective projects were moderately successful 
during their first year but showed a level of cost effectiveness that made it 
difficult for other, similar school districts to adapt the projects to their needs. 
It was often uncertain whether the unsuccessful districts could maintain their 
programs at the close of the project, and only slight intent was shown to do so. 



The average cost per student in the least effective projects during 

year was from $501 to $750, and the average increase in achievement w,a^ from 
six to ten months. Therefore, the calculated cost per student per monthbf — ^ 
growth was more than $50. Ilie relationships between program, evaluation, 
project objectives, and expenditures were vaguely indicated. The programs 
were barely effective in improving the achievement level of students, with a 
degree of cost effectiveness best described as poor. , 



Standardized test results, as reported for each project, were tabulated 
and compared to show increases in achievement in reading and mathematics. 
Achievement increases were expressed in terms of months of gain in mean 
grade placement for each month of instruction. Achievement was measured 
by the Comprehensive Tests of Basic Skills , Level 3, forms Q or R. This 
was administered as the pretest and post-test in all projects. Each project 
was rated separately for reading and mathematics. 

As shown in Table 2, projects rated highest in mean achievement gains in 
reading were Colton, OaUand, Santa Barbara, and Los Angeles (Edison). 
Projects reporting.lowest reading gains were Stockton, San Diego, Los Angeles 
(Belvedere), and Montebello. 

In mathematics, the projects rated highest in mean achievement gains were 
Los Angeles (Pacoima), Colton, Pasadena, and Long Beach. The mathematics 
project showing the lowest achievement gain was San Francisco, followed by 
Stockton, Los Angeles (Belvedere), and Riverside. 

To compare projects in which the number of months of instruction varied, a 
ten- month school year was considered the basis for determining instructional 
gain. Each monthly gain ratio reported was therefore converted to a yearly 
gain figure; for example, Colton's reported gain of 1.9/1 (1.9 months' gain per 
month) was converted to 19 months' gain for the school year. Similar figures 
were derived by multiplying each gain ratio in Table 2 by ten. 




Student Achievc^inent 
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Cost Analysis 

Coat- analysis ratings were compiled for per- student expenditures and 
cost effectiveness. Cost effectiveness was defined as the greatest increase 
in student achievement for the least cost per student. Projects were rated 
separately on reading and mathematics, with the least cost receiving the 
highest rating. ^ V 



Per ‘Student Expenditures 



Expenditures were summarized as reported. Costs were tabulated as 
of April 30, 1971, for the separate categories of (1) reading instruction; 

(2) mathematics ^instruction; (3) design and implementation (reported as 
"research and development"); ( 4 ) operating expenses; and (5) total expendi- 
tures per student. Projects were compared and ranked in order in each 
category of expenditures. These rankings are presented in tables 3 through 7. 

Table 3 reveals that the projects with the least expenditures per student for 
reading instruction were Colton, Montebello, Los Angeles (Belvedere), and 
Los Angeles (Pacoima). Projects with the greatest expenditures were Long 
Beach, El Monte, Oakland, and Santa Barbara. The per-student expenditure 
in the "most expensive" project was 11 times that of the "least expensive" 
project. 
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Table 4 shows that projects with the least expenditures per student for 
mathematics instruction were again Colton, Montebello, Los Angeles (Belvedere), 
and Los Angeles (Pacoima). Projects reporting the greatest expenditures were 
Long Beach, Oakland, San Jose, and San Francisco. The average per-student 
expenditure in the "most expensive" project amounted to ten times that of the 
"least expensive" project. 

Design and ipiplementation costs ranged from $3 to $508 per student. 

Table 5 indicates that these expenditures were lowest in the Stockton, Pasadena, 
San Francisco, and Colton projects. They were highest in the San Josci, Long 
Beach, and San Diego projects. 

Table 6 shows that the projects reporting the lowest per-student operating 
expenses were Colton and Montebello, followed by Riverside and Los Angeles 
(Belvedere). The highest operating expenses were recorded for Oakland, Long 
Beach (reading), San Francisco, and El Monte. Operating expenses for Oakland 
were 9.4 times those for Colton and Montebello. 

As shown in Table 7, total expenditures per student were lowest in Colton, 
Montebello, Pasadena, and Los Angeles (Belvedere). Greatest expenditures 
were reported for Oakland, San Jose, and Long Beach. Total expenses ranged 
from $145 per student in the "least expensive" project to $1, 320 per student 
in the "most expensive" project. 



Cost Effectiveness 



Increase in achievement per student was compared with expenditures per 
student to give a measure ’of cost effectiveness, as operationally defined. 

Cost effectiveness was expressed in two ways: (1) the number of months of 
achievement attained for each dollar spent per student; and (2) the cost in 
dollars per student of each month of achievement attained. 

The cost effectiveness of projects in terms of per- student expenditures 
for reading instruction appear in Table 8 and for mathematics instruction 
in Table 9. Cost effectiveness data for design and implementation are given 
in tables 10 and 11, for operating expenses in tables 12 and 13, and for total 
expenditures in tables 14 and 15. 

Table 8 reveals that the cost effectiveness of reading instruction ranged 
from one month of achievement for $3. 74 per student (Colton) to one month 
of achievement for $79 (Long Beach). Table 9 shows that the cost effective- 
ness of mathematics instruction ranged from Colton's $3.95 per student for 
each month of achievement to San Francisco's $84.25 per student. 

^ Table 10 shows that the cost effectiveness of design and implementation 
in reading projects ranged from 38 cents per student for each month of reading 
achievement (Stockton) to $39.08 per student (San Jose) Table 11 shows that 
the cost effectiveness of design and implementation in mathematics projects 
ranged from 33 cents per student for each month of mathematics achievement 
(Stockton) to $36. 29 per student (San Jose). 

. . " h 

!{ 

As indicated in Table 12, the cost effectiveness of operating expenses in 
reading projects ranged from one month of reading achievement for $6.89 
per student (Colton) to one month of achievement for each $79 (Long Beach). 

The cost effectiveness of operating expenses in mathematics projects, as 
shown in Table 13, ranged from Colton's $6.89 per student to San Francisco's 
$163.25 per student for each month of mathematics achievement. 

Table 14 reveals that the cost effectiveness of total expenditures in reading 
projects ranged from one month of reading achievement for each $7. 63 spent 
in Colton to one month for each $79 spent in Long Beach. Comparable figures 
for mathematics projects, as shown in Table 15, ranged from a cost of $7.63 
per student in Colton to $165. 75 per student in ^n Francisco. 

After the projects were compared and ranked in order on the four cost- 
effectiveness measures, the median rank order and the composite rank of the 
projects were calculated. The median rank order assigned was the median of 
the ranking of each project on the four; measures listed. The composite rank 
was the relative rank order of projects on median rank orders. The composite 
rank order of the projects on cost effectiveness is given in Table 16 and Table 17. 
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Table 16 shows that reading projects rated highest with respect to cost effective- ' 
ness were Colton, Montebello, Los Angeles (Pacoima), and Los Angeles (Edi'son). j 
Readiriig projects rated as least cost effective included Long Beach, Oakland,/ i 

San Jose, and Fresno. 
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The rank order of mathematics projects on cost effectiveness, given in 
Table 17, shows that the most cost-effective projects were Colton, Los 
Angeles (Pacoima), Montebello, and Pasadena. ^ Least cost effective were 
San Francisco, Oakland, San Jose, Stockton, and Long Beach. 



Combined Rank Order Rating of Projects 

Following the suggestions of the State Advisory Committee on Program and 
Cost Effectiveness, projects were rated on overall effectiveness, combining 
their rank-order ratings on three factors weighted as follows; achievement 
gains, 40 percent; operating costs per student, 40 percent; and program effective- 
ness, 20 percent. The rankings tl}at were combined to give the rating on overall 
effectiveness are presented in tables 18 and 19. 

Table 18 indicates that the reading projects rated highest in overall effective- 
ness were Colton, Santa Barbara, Riverside, and Montebello; projects rated 
lowest were Los Angeles (Belvedere) and Fresno, then Stockton, Long Beach, 
and San Diego. 

Mathematics projects ranked highest in overnil effectiveness, as shown in 
Table 19, included Colton, Riverside, [^Long Beach, and Montebello; projects 
rated lowest were San Francisco, Fresno, Lcs Angeles (Belvedere), and 
Stockton. ' 

It was agreed that no project could be considered cost effective that was not 
first of all effective in improving achievement. The criterion for effectiveness 
in achievement gains was arbitrarily set at the median gain for all projects, 
as reported. It was also agreed that projects falling in the lowest quarter on 
several separate ratings should be considered least effective. Quartile rankings 
of projects are summarized in Table 20. 

It was further agreed that program effectiveness should be considered . 
separately from cost effectiveness in determining the least effective projects. 
Projects were ranked in order of overall effectiveness and in order of cost 
effectiveness, as defined. 











Evaluation Results 



Evaluation results reveal that the projects rated highest on program 
effectiveness were Santa Barbara^ El Monte, Colton, Riverside, and Long 
Beach (mathematics). Projects rated lowest in program effectiveness 
included Fresno, Pasadena, Los Angeles (Pacoima), and Los Angeles 
(Belvedere). \ 

Reading projects rated highest on student achievement in reading were 
Colton, Oakland, s|nta Barbara, and Los Angeles (Edison). Projects 
rated lowest on acmevement in reading were Stockton, San Diego, Los 
Angeles (Belve der/ ). and Montebello. 

Mathematics projects reporting highest student achievement included 
Los Angeles (Pacoima), Colton, Pasadena, and Long Beach. Mathematics 
projects rated lowest in achievement were San Francisco, Stockton, Los 
Angeles (Belvedere), and Riverside. 



Projects reporting lowest operating expenses per student were Colton, 
Montebello, Riverside, and Los Angeles (Belvedere). Highest per- student 
operating costs were recorded for Oakland, Long Beach (reading), San 
Francisco, and El Monte. 

Lowest total expenditures per student were reported by Colton, Montebello, 
Pasadena, and Los Angeles (Belvedere). Highest expenditures per student 
were for Oakland, San Jose, and Long Beach. 



Reading projects rated highest on cost effectiveness were Colton, Montebello, 
Los Angeles (Pacoima), and Los Angeles (Edison). Reading projects rated as 
least cost effective included Long Beach, Oakland, San Jose, and Fresno. 

// • 

Mathematics projects rated highest on cost effectiveness included Colton, 

Los Angeles (Pacoima), Mor/cebello, and Pasadena. The least cost-effective 



mathematics projects were 
Long Beach. 



:;!an Francisco, Oakland, San Jose, Stockton, and 



Reading projects rated highest in overall effectiveness were Colton, Santa 
Barbara, Riverside, and Montebello. Those rated lowest included Los Angeles 
(Belvedere), Fresno, Stockton, Long Beach, and San Diego. 



Mathematics projects rated highest in overall effectiveness included Colton, 
Riverside, Long Beach, and Montebello. Rated lowest in overall effectiveness 
were San Francisco, Fresno, Los Angeles (Belvedere), and Stockton. 



Projects ranking above the highest quartile most often were Colton and 
Riverside. Projects placing below the lowest quartile most often were San 
Francisco and Los Angeles (Belvedere). 
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On the basis of these evaluations, it was recommended that (1) the most 
effective demonstration programs should be retained, extended, and com- 
mended: (2) effective programs should be retained, improved, and strengthened: 
and (3) the least effective programs should be terminated. 
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4.5 
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BANK ORDER OF FROJBCTS ON MRAM AClXIflMMI GA»S 
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Rank 


2Kfilt£l 


■Mhor of Maatha of 
Aclitaoaoaot la Raadiat 
Far lach Dollar Spoat 


Coat la Dallara Far Student 
of lach Month of 
AehiavaaMnt in' Raading 


1 


Colton 


0.27 , 


$ 3.74 


2 


Montabollo 


0.10 


9.56 


3 


L.A. - Pacolaa 


0.09 


11.31 


4 


L.A. - Idiaon 


0.08 


12.07 ' 


5 


Rivaraida 


0.07 


15.38 


6 


L.A. - lalvadara 


0.06 


! 

16.11 

} 


7 


li, 

San Praneiaeo 


0.04 

.'.j 


25.00 1 

i 


8 


Santa Barbara 


0.03 


33.38 ; 


9 


Stockton 


0.03 


34.00 

1 


10 


Fraano 


0.03 


38.10 j 

1 


11 


San Joaa 


0.03 


38.77 i 
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Oakland 


0.03 


38.82 1 


13 


San Oiafo 


0.02 


58.00 1 


14 


11 Manta 


0.01 


66.82 ( 
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Long loach 


0.01 


79.00 i 
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COST EFFECTIVENESS OF MATH PROJECTS . , 

IN TERMS OF PER-STUDENT EXPENDITURES FOR 
MATHIMATICS INSTRUCTION 



Rank 


Proicct 


Kunber of Months of 
Achievement in Math for 
Each Dollar Spent ’ 


Cost in Dollari Per Student 
of Each Month of 
Achievement in Math 


1 


Colton 

• * 


0.25 


$ 3.95. 


2 


L*A. - Pacoina 


0.14 


7.00 


3 


Montebello 


0.11 


8.90 


4 


L*A. - Edison 


0.07 


15.36, 


5 


Pasadena 


0.06 


15.76 


6 


L*A* - Belvedere 


0.06 


16.11 


7 


Riverside 


0.04 


22.89 


8 


Fresno 


0.04 


23.93 


9 


Stockton 


0.03 


30.22 


10 


San Jose 


0.03 


' 36.00 


11 


Oakland 


0.C2 


47.14 


12 


Long Beach 


0.02 


48.06 


13 


San Francisco 

r / 


0.01 


\\ 84.25 
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TABU 10 



COST EFFECTIVENESS OF READING PROJECTS 
IN TERNS OF PER-STUDENT EXPENDITURES FOR 
DESIGN AND DgLEMEMTATIOH 



Rank 


Project 


Mua^r of Months of 
Achiavaoent in Reading 
For Each Dollar Soent 


Cost in Dollera Per Student 
of Each Month of 
Reading AchlavaaMnt 


1 


Stockton 


2.67 


$ 0.38 


2 


Colton 

f 


u • 

1.27 


, : 

0.79 


3 


Son Francisco 


0.87 


1.15 


4 


L.A. • Pacolma 


0.48 ( 


2.08 

3.21 


5 


L«A. - Edison 


0.31 


6 


L«A. - Belvedere 


0.23 


4.44 


7 


Montebello 


0.20 


4.89 


8 


Oakland 


0.19 


5.24 


9 


Santa Barbara 


0.16 


6.13 


10 


El Monte 


0.10 


10.00 


11 


Fresno 


0.09 


11.60 


12 


Riveralde 


0.08 


11.92 


13 


San Diego 


0.05 


22.00 


14 


Long Beach 


0.04 


27.80 


15 


San Joaa 


0.03 


39.08 
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TAILE 11 

COST EFFECTIVENESS OF HATH PROJECTS 
IN TERMS OF PER-STUDENT EXPENDITURES FOR 
DESIGN AND IMREMEWTATION 



Rank 


Pro lee cl 


Number of Months of 
Achievement In Math 
for Each Dollar Spent 


Cost In Dollars Per Student 
of Each Month of 
Achievement In Math 


1 


Stockton 


3.00 


$ 0.33 


2 


Pasadena 


1.42 


0.71 


3 


Colton 


1.27 


0.79 


A 


L.A. - Pacolma 


0.78 


1.29 


5 


San Francisco 


0.27 


3.75 


6 


li«A* * Edison - ' 


0.24 


4.09 


7 


Montebello 


0.23 


< 4.40 


8 


L.A. - Belvedere 


0.23 


4.44 


9 


Oakland 


0.16 


6.36 


10 


Fresno 


0.12 


8.29 


11 


Long Beach 


0.07 


13.43 


12 


Riverside 


0.06 


17.22 


13 ' 


San Jose 


0.03 


36.29 
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TABLE 12 



COST EFFECTIVENESS OF READING PROJECTS 
IN TERMS OF PER-STUDENT EXPENDITURES FOR 
OPERATING EXPENSES 



Rank 


Project 


Number of Months of 
Reading Achievement 
for Each Dollar Soent 


j 

Cost in Dollars per Student! 
of Each Month of [ 

Reading Achievement f 


1 


Colton 


0.lsS‘) ■ 


$ 6.89 j 

* i 


2 


Montebello 


0.07 


14.56 j 

1 


3 


Riverside 


0.05 


19.31 1 


4 


L.A. - Pecoima 


0.05 


20.54 i 

1 


5 


L.A. - Edison 


0.05 


20.93 1 

f 


6 


Santa Barbara 


0.04 


27.25 


7 


L.A. - Belvedere 


0.04 


28.00 1 


8 


San Diego 


0.03 


36.00 1 


9 


San Jose 


0.03 


38.46 1 


10 


San Francisco 


0.02 


50.23 


11 


El Monte 


0.02 


56.91 


12 


Fresno 


0.02 


59.90 


13 


II Stockton 

• } t 


0.01 


67.63 


14 


V Oakland 


0.01 


72.41 


IS 


Long Beach 


0.01 


79.00 
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/ COST EFFECTIVENESS OF MATH PROJECTS 

IN TERMS OF PER-STUDENI EXPENDITURES FOR 
OPERATING EEPENSES 



Rank 


Project 


Nnhber of Months of 
Moth Achieveeient 
for Each Dollar Soent 


Coat in Dollars Per Studei 
of Each Month 
of Math Achleveaient 


1 


Colton 


0.15 


$ 6.89 


2 


L.A. - PocolM 


0.08 


12,71 


3 


Montebello 


0.08 


13.10 


A 


Peeadene 


0.07 


15.12 


5 

f 


L«A< “ Edleon 


0.0/.^ 

\\ 


26.64 


6 


Xlvereide 


\ 0.04 


27.89 


7 


L.A. - Belvedere 


0.04 


28.00 


8 


Long Beech 


0.03 


30.38 


9 


San Joee 


0.03 


35.71 


10 


Fresno . 


0.02 


42.79 


11 


Stockton 

• / 


0.02 


60.11 


12 


Oakland 


0.01 


67.93 


13 


San Franciaco 


0.01 


163.25 
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COST EFFZCIIVEMBSS OP UADING PROJECTS 
IN TERMS OF PBR-STUDEIfT 



Rank 


Proinet 


Mneber of Montha of 
Eaadlng Aehlavenent 
for Each Dollar Spent 


Coat in Oollara Par Student 
of Each Month of 
AchlevMant in Reading 


1 


Colton 


0.13 


1 7.63 


2 


Montabnll.0 


0.05 


19.4A 


3 


L.Ai - Pncoina 


0.04 


22.62 


4 


ItftAt " Edlaon 


0.04 


24.14 


5 


LiAi - Bolvadtro 


0.03 


32.33 


6 


Santa Barbara 


0.03 


33.38 


7 


Eiveralde 


0.03 


. 37.38 


8 


San Franciaeo 


0.02 


31.00 


9 


San Diego 


0.02 


38.13 


10 


El Monte 


0.01 


66.82 


11 


Stockton 


0.01 


67.88 


12 


Presno 


0.01 


71.60 


13 


San Joe a 


0.01 


77.54 


14 

r 

JU 


Oakland 

. . ■■ 


0.01 


77.65 


Long Beach 


0.01 


79.00 
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TABLE 15 






COST EFFECTIVENESS OF MATH PROJECTS 
IN TERMS OF PER-STUDENT 
TOTifl, EXPENDITURES 



Rank 


Proleet 


Nunbcr of Months of 
Math Achievement 
for Each' Dollar Spent 


. Cost in Dollars Per Student 

of Each Month of | 

Achlevaiant in Math 1 


1 


Colton 


0.13 


ir, 

$ 7.63 ^ 

'ii 


2 


,L.A. - Pacolna 


. 0.07 


14.00 1 


3 


Paaadena 


0.06 


15.76 


4 


Montebello 


0.06 


17.50 


5 


L*A« * Edlaon 


0.03 


30.73 

j 


® r 


L.A. - Belvedere 


0.03 


32.33 


7 


Long Beach 


0.02 


48.00 


8 


Freano 


0.02 


51.14 


9 


Riverside 


0.02 


54.00 


10 


Stockton 


0.02 


60.33 


11 


San Jose 


0.01 


72.00 


12 


Oakland 


0.01 


94.29 


13 


San Francisco 


0.01 


165.75 
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Colton 
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A* 

2 


1* 


1 


1 


1 


Mentnbtllo 


2 


7 


2 


2 


2 


2 


L«A»*Pocolnn 


3 


4 


A 


3 


3.S 


3 


L.A.-Ulton 


4 


5 


S 


4 


4.5 


4 


L.A.-Bolvodoro 




« 


7 


S 


• 


s.s 


Klvoroldo 


S 


12 


3 


7 


4 


s.s 
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Appendix B 

A Summary of the Most Effective Project: 

Demonstration Programs in Reading and Mathematics 
at Colton Junior High &hool 

■•i 

As documented in their 1970>71 annual report, the demonstration programs 
in reading and mathematics at Colton Junior High School were highly success- 
ful. ^ The following is a brief summary of the planning, program, and evalua- 
tion activities that resulted in the highest level of student performance and 
cost effectiveness of any demonstration project in California during 1970-71. 



Project Planning at Colton 

Colton Junior High School's demonstration project was planned in December, 
1969, and implemented during the 1970 spring semester. It was designed 
primarily as a three-year spiraling program, with student skills built one upon 
the other to achieve the following goals: (1) allow students to assume more of 
the responsibility for their own education; (2) improve academic achievement in 
all subjects by improving reading and mathematics skills; and (3) improve 
student attitudes. 

The original project proposal was written by a team of teachers, adminis- 
trators, and school district personnel. This team approach lias been continued 
through each year of the project's operation. There were two instructional 
teams, one composed of four reading teachers and one composed of four math 
teachers. These two groups scheduled, planned, and evaluated activities together. 
The project director scheduled approximately 120 students for each period of 
mathematics and each period of reading; then the teams assigned students to 
individual classes. 

Three basic types of learning arrangements were used extensively in the 
Colton project: large group instruction, small group discussion, and inde- 
pendent study. 



, . Program Innovations in the Colton Project 

Since the reading program was ungraded and utilized miniunits, students 
were assigned to individual classes on the basis of interest and diagnosed needs. 



^"AB-938 Demonstration Program, Second Annual Report. " Colton, Calif. ; 
Colton Joint Unified School District, June 10, 1971 (mimeographed). 
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Miniunits were offered at five- or six-week intervals, and both students and 
teachers were allowed to sign up to study (or teach) the topics that appealed 
to them. Some of the units included during 1970-71 were propaganda devices, 
the newspaper, detective fiction, good grooming, motorcycles, vocations, 
scriptwriting, and reference skills. . Paperbacks, field trips, reading machines, 
filmstrips, speakers, movies, newspapers, and teacher- prepared materials 
were used to help individualize the units. 

The reading laboratory was used for a variety of instructional purposes, 
including remediation and teaching of the miniunits. The lab was equipped 
with controlled readers and accompanying filmstrips, listening centers,^- 
individual study carrels, tape recorders, a language master machine, and 
film equipment. Software items included programmed reading workbooks, 
paperbacks, Sullivan materials, selected st<H*ies, textbooks and brochures 
for individual study, and teacher-prepared pamphlets. 

The thathematics program was not ungraded; team members met after 
the diagnostic testing, however, and assigned students to the instructors who 
stressed the areas of student need indicated by the tests. This team also 
prepared its own booklets and teaching materials so that students could 
progress at their own speed and work on the units they particularly needed. 

The math booklets emphasized basic skills, so that students could enter the 
regular state-adopted mathematics program as soon as possible. 

Both teams met weekly for planning, inservice training, discussion, and 
evaluation. The project director, who was also the school principal, scheduled 
all teachers of a subject to the same conference period for these weekly 
meetings. 



Continuous Evaluation 

Chitside evaluation was provided quarterly to give direction and recommenda- 
tions and to assess the attainment of goals and objectives. Student and faculty 
interviews, questionnaires, observations, and tbst data were included, based 
upon weekly visits of the project evaluator. 

Both standardized and nonstandardized tests were used in the evaluation. 
Standardized test data indicated significant gains in reading comprehension, 
vocabulary, and total reading scores. Student achievement in mathematics 
included significant gains in computation, concepts, and applications and in 
mathematics as a whole. 

Colton's unique utilization of the outside project evaluator for weekly obser- 
vations and quarterly reviews was outstanding. G. Keith Dolan of California 
State College, San Bernardino, visited the project each week and gathered data 
for program improvement. He presented these data to the project staff in 
writing once each quarter. The project staff replied with quarterly written 
reports, listing the actions they had taken in response to the evaluator's 
recommendations. 
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Program Revisions at Colton 

After the evaluator's first quarterly report in November, 1970, the Colton 
staff listed program revisions in 18 areas, including those identified in the 
paragraphs that follow. 

Additional inservice training time was devoted to individualized mathematics 
instruction, including released time for teachers to visit other schools; consul* 
tations with personnel in offices of county superintend jints of schools andiWork- 
shops with state college professors; use of SRA mathematics tapes; review of 
new materials; and individual teacher conferences. Instructors exchangev^ 
students to capitalize on individual teacher strengths. Teachers alternated 
their teaching strategies. The time of three teacher aides involved in the 
project was rescheduled to permit more individual tutoring, including four 
periods per day in the classroom. Classified employees were reevaluated 
to make full use of personnel strengths. Teacher aides were included in 
team planning meetings. Additional volunteer help was secured through the 
use of eighth grade students and student teachers to assist with reading. 
Mathematics goals were rewritten in terms of specific student performance 
and behavioral objectives. The use of parents as tutors outside the classroom 
was studied. The objectives of the counselor were redefined, and priorities 
were assigned to individual counseling and guidance. A test-scoring device 
was purchased to assist the staff with test correcting. General program 
objectives were restated in behavioral terms. 

In addition, the meaning of individualized study trips was clarified. (The 
term "individualized study trips" means that only those students who are 
interested in the subject participate; it does not mean that only one or two 
students participate. ) A calendar of study trips was prepared by the instruc- 
tional teams. Guest speakers were scheduled for classroom visits. New 
instructional miniunits in reading were written and prepared for subsequent 
class sessions. Regular weekly staff meetings were scheduled separately ' 
for reading and mathematics teams. A student advisory committee was estab- 
lished. Efforts were directed toward expanding the group of parent volunteers, 
as recommended by the student advisory committee. Thank you notes and 
Christmas cards were sent in recognition of the efforts of the parent volunteers, 
and congratulatory notes were sent to the parents of students on the student 
advisory committee. A newsletter was published and sent home. Dr. Dolan 
presented a summary of his quarterly evaluation to the school faculty meeting. 
Inservice sessions were scheduled for regular staff meetings. Reading teachers 
reported on conferences attended. Regular biweekly meeting times and dates 
were established for the student advisory committee. Action was taken to 
remedy the weaknesses of the programs, as identified by the staff evaluation 
questionnaire -- weaknesses in communications, inservicc training, use of 
planning time, and articulation with the senior high school. 
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**Conmimer Reports** 



Students reported their liking for the electives, the a.j^er school activities, 
the teachers, the "work-at>your*own-speed" booklets, the miniunits, and 
the eight different periods. They felt that some booklets were too long, some 
classes too short, and some vocabulary words too hard and that there were 
not enough paperlracks in the reading classes. 

Reading teachers reported that the most effective elements of the program 
included the team effort, behavioral objectives (a better way to teach), 
miniunits, we eMy planning meetings, qualified teacher aides, paperbacks 
and magazines, and* the reading bpoldets. Mf^thematics teachers felt that 
the most effective elements of the program ipcluded the teacher aides, ' individu- 
alized teaching, the variety of materials, diagnostic testing and prescriptive 
planning, and the math booklets. 

A principal's advisory group felt that the most effective elements of the 
program were individualized teaching, reading instruction, community aides, 
use of paperbacks, the miniunits, and the outside evaluator. 



Final Evaliation,of the Project 

During the project year the project staff directed considerable effort to 
follow through on all the suggestions made by the project evaluator. From 
his point of view, "These dedicated efforts have done much to improve the 
project during the past several months." 

In a final student evaluation of the project, there was a strong positive tone 
to thb comments made, with favorable comments outnumbering the unfavorable 
by two to one. More than 64 percent of the students considered the program 
to be helpful, especially in reading. Positive comments referred most often 
to such components as the miniunits, "learning at my own rate," the use of 
paperbacks, and the variety of teachers and classes. 

A final parent questionnaire revealed that the parents were overwhelmingly 
supportive of both the mathematics and reading programs, with nearly 90 percent 
of the respondents indicating that they would like to have their children continue 
in the programs. All but one parent considered the programs to be at least 
"moderately effective." 

Overall, the results of the formal evaluation, student achievement data, 
behavioral objectives, and evaluator's surveys of faculty, students, and parents 
indicated a highly successful demonstration program in Colton for 1970-71. 



Conclusion to the Colton Project 

In an attempt to answer the question of how to obtain the greatest increase 
in student achievement for the least cost per student, this summary of the 
program and evaluation activities of the Colton project seems to lead to one 
conclusion: Improvement in student achievement appears to be more closely 
related to such factors as individualized instruction, use of local resources, 
teacher aides, a variety of teacher-made materials and methods, continuous 
evaluation, and parent- teacher- pupil interaction than it is to such costly items 
as plant remodeling, equipment, hardware, or technical assistance. 
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Appendix C 

Education Code Sections on the Subject 
of Demonstration Programs 



h 

Artiel* 5. DmonitratiOB ProfitM is Intradf* iBi t me ttos 

in Reading and Mathematics for Lott Achieving Pupila 
(Article 5 added bjr Stats. 1966 (1st Ex. Bess.), Ch. 106. Effcc* 

tire ontil 91st day aftec adjournment of 1972 Regular Sea* 

•ion) 

tagfiloNva fnlant 

6490. It ia tha iatant asd porpoaa of tha Lai^taia that 
axamplary prograaM ba aatablibbad for Istaaaiva iaatrsetka 
is raading and BaathaaMtica to aarvo aa daoMaatratloa piojooli 
aioMd aoldy at dorolopiac, witbls pspiK abmamap aaai> 
patasea ia thcaa baaic akin anb jaeta. Tha program Aall ba 
daveloped to aerva pnpils in grada 1, 8, or 9 who attend school 
in designated araas of disadvaataga, aad who otherwise would 
&d difflenltgr in aehieriBg eompleta tneeeas ia high se^t 

It is tbo further intent of tha Legialatnra that these pro* 
grams ia intensiva iastructioa in reamag aad aaatiiamaties be 
operated by school districts direeUy, or by school districts 
through eoBtrset for partial or complete oparatioo with aay 
competent public or private agency, foundation or corporation. 
It is also tha intent of the Legislature that authority be 
gnnted to permit tha waiver of any provision of the Education 
&>de bf tlM program approving agency, if such is ncecsEary 
for tha development of aiodel ^temonstration progranu ia tha 
intensive instruetion ia readbig and matiiematies. 

(Added by Stats. 1966 (1st Ex. Sees.), Ch. 106; amended in 
identical language by Stats. 1969, Ch. 1578 and Ch. 1596. Effec- 
tive September 6, 1969, until 91st day after adjournment of 
1972 Regular Session.) 

AidmIhMntion and Apport l onmant of Fvndii 

6491. From the funds appropriated therefor by tha Legida- 
tore to the Department of Bducstioa for the purpoaes of this 
article, ^ Superintendent of Public Instruction, upon recom- 
mendation of the Director of Compensatory Education, with 
the approval of the State Board of Bdneation, shall administer 
this article and make apportionments to aehooi districts to meet 
the total approved expose of the sehool districts incurred in 
establishing demonstration prograsu in the intensive instruo- 
tion in reading and mathematics for pnpils in grades 7, 8, or 9. 

(Added by Stats. 1966 (1st Ex. S^), Ch. 106; amended in 
identical language by Stats. 1969, Ch. 1578 and Ch. 1596. Effec- 
tive September 6, 1969, until 91st day after adjournment qf 
1972 Regular Session.) 
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Appfieofien for Program by Diilriel 

6492. The gorerning board of any diitriet which 

grades 7, 8, or 9 on acconnt of anj school or sehoola located 
la aaj area designated by the Director of Compenaatoiw Eda* 
cation parraant to the prorisiona of Edaeation Code Seetion 
6482, may make application to catabliah and operate a progrnra 
under this article. The application shall be in the form er>d 
shall contain such data and information as the director shall 
specify. 

(Added by Stats. 196iC (1st Ex. Sees.), Ch. 106; amrnded in 
identical langua^'e by Stats. 19C9, Ch. 1578 and Ch. 1596. EfTce- 
live Septfinbcr 6, 1969, until 91st day after adjournment of 
1972 JiegulMr Session.) 

WeiVer of ProWiiont of this Cede 

6493. The goreming board of a school district, in its ap> 
plication, may r^uest waiver of the provisions of any section 
.or sectiona of this code if such waiver is necessary to e.'*.abUsh 
a:>.d operate a program under this article. The need for a 
waiver shall be explained and justified in the application. The 
Superintendent of Public Instruction, upon recommendation 
of thn Director of Compenantory Education, with ^e approval 
of the State Board of Education, may grant, in whole, or in 
part, sny such reciuest 

(Added by Stats. 1966 (1st Ex. Sen.), Ch. 106. Effective 
until 91st day after adjournment of 1972 Regular Session.) 

OperoNon of Programs 

6494. Tha govaming board of tho achool diatrict may make 
application to esteblieh and operate a demouatratioB program 
under thia article directly, or mey uteke applicatiou to provide 
for partitf or eompltte operation of auch a program through a 
contract with any eoippetent public or private agency, founds 
. tion or corporation. 

(Added by State 1966 (lat Ex. Seas.), Ch. 106. Effective 
yntil 91st day after adjournment of 1972 Regular Session.) 

Stondorefs and CrHaria by Stata Board of CdveaHon 

6495. The State Board of Education ihall adopt regulations 
setting forth the standards and eriteria to be used in the evsln- 
Btion of spplicBtions submitted by school districts. The stand- 
srds snd criteria adopted by the State Board of Education, 
among other items, shsll inelnde s statement of specific goals 
to I'-' sought in the progrsm both in terms of pupil achieve- 
ment snd for the purpose of estsbliehing s model program, snd 
tho requirements for evslnstion of the progrsm. 

Projects shsll be siq>roved only if it ean be shown that, if 
suceei^ul, the cost effeetivencM of the project wiU be auch as 
to be adaptable within the budgets of other similar school dte 
trieta throughout the state. 
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Pro jeeti ihaU bo eontinoaUj roriewtd rogordiaf thoir eiee> 
ttroBoai in iaproring tho iebMYcmont loveb of pqtUs in rood* 
inf and mathenntieo. Projeeti ivbieb aro least coat offaetivo 
ahall ba tarainatad and ahall ba raplaead with oom of pio?an 
affaetiTaneoB or bj now projaets i^b bdd proatiaa of in* 
eraased effeetiTanaoB. 

(Added bjr Stats. 1966 (lit Ex. Seas.), Ch. 106; amended in 
identical langnafo bj Stats. 1969, Ch. 1578 and Ch. 1596. Effee* 
tive September 6, 1969, until 91at day after adjournment of 
1972 Regular Seaaion.) 

Carfiflcofton of ond Amount of Apport i onmut 

6686^ Upon ^roral h7 the State Board of Ednaation of 
an appUeattoa uiidiv this artiola, tho Sapariataadant of Pnblia 
laitnietioa Aall eartifjr ^ aawuit to ba apportkaad to tho 
applicant acbool diatriot. 

(Added by SUta. 1966 (lit Ez. Sem.), Ch. 106. Effective 
until 91at d^ after adjournment of 1972 Regular Seaaion.) 

Efohfotlon depoft 

6497. No later than the fifth legialatiTe day of each regnlar 
aeaaion of the Legialatnra, the Superintendent of Public 
Inatruetion, upon recommendation of the Director of Com* 
penaatory Education, with approval of the State Board of 
Education, ahall submit a report to the Legislature on the 
implementation and evaluation of demonstration programs 
under this article, including the achievement of pupils, an 
analysis of the costs of each project detailed in ternu of the 
costs of design, implementation and continuing operational 
expenses, ineinding the degree of cost effeetiveneae of each 
project The report ahall include recommendations con* 
cerning improvement retention, extension or other aspects of 
theprograBL 

(Added by Stats. 1966 (1st Ez. Sees ), Ch. 106; amended 
in identical language by Stats. 1969, Ch. 1578 and Cli. 1596. 
Effective September 6, 1969, until 91st day after adjournment 
of 1972 Regular Seuion.) 

T sm W weWea of Effect of This Aitkio 

6498. This artielo ahaU have no force or offset after tho 
91st day following the adjourmnent of the 1972 Regular Sea* 
akm of the Legialatnra. 

(Added by Stats. 1966 (1st Ex. Seas.), Ch. 106; amended 
by Stats. 1969, Ch. 1578 and Ch. 1596. Effective September 6, 
1969, until 91st day after adjournment of 1972 Regular Ses* 
sion.) 
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ABSTRACT 

The purpose of the demonstration programs in 
intensive reading and mathematics instruction for low— achieving 
students was essentially to enable school districts to establish and 
operate exemplary and innovative projects to improve the competence 
of junior high school students in reading and mathematics. Projects 
were limited to students in grades seven, eight, and nine attending 
schools located in low-income areas. Participants were educationally 
disadvantaged students who would otherwise find difficulty in 
achieving success in high school. Project proposals were required to 
contain statements of specific goals with respect to student 
achievement and to show a level of cost effectiveness that would make 
it possible for other, similar school districts in California to 
adapt the projects to their needs. Projects that proved least cost 
effective were to be terminated. Seventeen projects were approved and 
1970—71 was their second year of operation. Projects were continually 
reviewed by the Division of Compensatory Education regarding their 
effectiveness in improving the achievement level of students in 
reading and mathematics. Through reports, questionnaires, 
observations, and interviews, projects were evaluated on the basis of 
several criteria involving program development, student achievement, 
and cost analysis. (Author/JM) 
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Overview of the Demonstration Project 



The California State Department of Education is in the process of creating 
more accountable educational programs throughout the state. The project 
described in this report - -a report required by Education Code Section 6497 -- 
is a step toward a model of accountability. 

With the passage of Assembly Bill 938, the 1969 Legislature authorized the 
Division of Compensatory Education to establish demonstration programs in 
intensive reading and mathematics instruction that would improve the academic 
achievement of low-achieving students and be cost effective. Demonstration 
programs were established in 17 of the poorest schools in California with 
respect to their students' socioeconomic status and academic achievement. 

In terms of educational failure and lack of hope, these were the schools that 
had the greatest need for such programs. 

The students in the most cost-effective demonstration programs have shown 
a rate of achievement seen in few compensatory education programs anywhere 
in the nation. These programs were designed with the objective of assisting 
students to attain a more normal range and distribution of achievement. This 
objective was achieved in almost all cases. The few programs that were not 
successful were terminated. 

The programs are unique in many ways. They are planned and developed 
by the staff of each participating school. The principal, the project director, 
and a carefully selected staff of teachers, aides, and volunteers make decisions 
at the school level. Students with varying abilities are grouped together 
heterogeneously. They are usually taught individually or in groups of two or 
three at least part of each day by teachers and assistants. In many programs, 
each student spends some time each week in a learning laboratory where he 
works with highly trained teachers and assistants in specific learning activities -- 
activities designed especially for him. 

Curriculum is developed and prescribed as needed and is based upon a careful 
diagnosis of learning disabilities. Success is the key to the program. The 
students are made aware of their successes rather than criticized for their 
failures. Students feel the high expectations and in turn are motivated to learn. 

The program attempts to remove many obstacles that hinder learning; for 
example, sections of the Education Code may be waived if their provisions seem 
to interfere with an innovative program. 

Another unique feature of the programs is that, unlike most other state or 
federal projects, those projects that are considered to be the least cost effective 
are terminated. Funds from, terminated projects are being used to replicate 
cost-effective programs in more schools within districts that have already 
conducted successful projects. 
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The demonstration programs began at the seventh grade level in 1969-70, 
followed the participating students into grade eight in 1970-71, and is serving 
them in grade nine in 1971-72. All eligible students at the appropriate grade 
levels in each project school were served. As the state-funded programs 
moved from one grade to the next, the school districts established similar 
programs for incoming students in the grade no longer being served by the 
original programs. Because of the success of their demonstration programs, 
several districts have begun similar programs in additional junior high schools 
within their districts, generally with tremendous impact on traditional instruc- 
tional programs in reading and mathematics. 



Implementation of the Programs 

The purpose of the demonstration programs in intensive reading and 
mathematics instruction for low-achieving students was essentially to enable 
school districts to establish and operate exemplary and innovative projects 
to improve the competence of junior high school students in reading and mathe- 
matics. Seventeen projects were approved, and 1970-71 was their second 
year of operation. 

Projects were limited to students in grades seven, eight, and nine attending 
schools located in low- income areas. Participants were educationally disad- 
vantaged students who would otherwise find difficulty in achieving success in 
high school. 

Project proposals were required to contain statements of specific goals 
with respect to student achievement and to show a level of cost effectiveness 
that would make it possible for other, similar school districts throughout 
California to adapt the projects to their needs. Projects that proved least 
cost effective were to be terminated. 

The most effective demonstration programs in reading were those in Colton, 

Los Angeles (Edison), Los Angeles (Pacoima), Riverside, and Santa Barbara. 

The most effective mathematics programs were in Colton, Los Angeles (Pacoima), 
Pasadena, and Riverside. 

Effective demonstration programs in reading included those in El Monte, 

Fresno, Montebello, Oakland, San Diego, San Francisco, and San Jose. 

Effective mathematics programs were implemented in Fresno, Long Beach, 

Los Angeles (Edison), Montebello, Oakland, and San Jose. 

The least effective demonstration programs included the projects in reading 
and mathematics at Los Angeles (Belvedere) and Stockton, the San Francisco 
mathematics program, and the Long Beach reading program. 
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Recommendations Regarding the Programs 

Considering the purpose of the programs as defined by Assembly Bill 938, 
evaluation criteria, and evaluation results, the following recommendations 
are offered regarding the demonstration programs in reading and mathematics: 



1. Demonstration programs in the following districts should be retained and 
extended; they should be commended as most effective; and efforts should 
be directed toward greater demonstration and dissemination of information 
regarding their most innovative and exemplary cost-effective elements: 

f 

Reading programs Mathematics programs 



Colton Joint Unified 

Los Angeles Unified (Pacoima) 

Riverside Unified 

Santa Barbara City Elementary 

Los Angeles Unified (Edison) 



Colton Joint Unified 
Los Angeles Unified (Pacoima) 
Pasadena Unified 
Riverside Unified 



, 2. The demonstration program in the Montebello Unified School District should 
be commended for its excellence, although the district did not apply for 
renev/al of the project for 1971-72. The division of Compensatory Education 
should continue further dissemination of information about the program and 
this district's outstanding replication of exemplary elements in the seventh 
and eighth grades. 

3. Effective demonstration programs in the following districts should be""^"^' 
retained and strengthened, and efforts should be directed toward modifying 
these programs to improve their program effectiveness and cost effective- 
ness and increase their achievement gains: 



Reading programs 

San Francisco Unified 
El Monte Elementary 
Oakland City Unified 
San Jose Unified 
San Diego City Unified 
Fresno City Unified 



Mathematics programs 

Long Beach Unified 

Los Angeles Unified (Edison) 

Oakland City Unified 

San Jose Unified 

Fresno City Unified 



4. Demonstration programs in the following districts should be terminated 
as least effective for the reasons indicated: 



a. Los Angeles Unified (Belvedere) (reading and mathematics). The Belvedere 
project rated low on organization and administration, program development, 
and fiscal management and lowest of all projects according to an overall 
evaluation questionnaire (rank: seventeenth). The project also rated low 
in project observation data, low on a visitation inventory, and low on overall 
program effectiveness (rank: fourteenth). The project ranked 12. 5 out of 
15 projects in reading achievement and eleventh out of 13 projects in mathe- 
matics achievement gains. The project ranked in the lowest quarter in 
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reading achievement, mathematics achievement, and program effective- 
ness. In overall rank order (weighted 40 percent achievement, 40 percent 
costs, 20 percent program). Belvedere ranked fifteenth out of 15 reading 
projects and 10.5 out of 13 mathematics projects. 



b. Stockton City Unified (reading and mathematics). The Stockton project 
tied another district for lowest place on the program visitation inventory 
(rank: 15.5 out of 16). The project rated low in project observation 
data and in overall program effectiveness (rank order: 11.5). Baseline 
data for 1969-70 revealed low ratings in the evaluation report, in reading 
achievement gains, and^in costs per student. The project rated lowest 
among 15 reading programs in reading achievement gains (0.8 year's 
growth per year) and rated s’econd lowest in mathematics gains (rank: 
eleventh out of 13). The project showed high operating costs (rank: 
tenth) and, in cost effectiveness, ranked ninth out of 15 reading projects 
and ninth out of 13 mathematics projects. It ranked in the lowest quarter 
on reading and mathematics gains and below the median on operating 
costs and program effectiveness. In overall rank order (weighted 40 percent 
achievement, 40 percent costs, 20 percent program), Stockton ranked 
twelfth out of 15 reading projects and 10. 5 out of 13 mathematics projects. 



c. San Francisco Unified (mathematics). The San Francisco mathematics 
program rated low on the program evaluation questionnaire (rank order: 
twelfth out of 17 programs). 'It rated lowest of all programs in achievement 
gains (rank: thirteenth) -- achievement gains reported were equal to four 
months per year of instruction. High expenditures per student were 
reported for operating costs (second highest, ranking twelfth out of 13 
mathematics programs) and for total expenditures per student. The 
program was the least cost effective of all 13 mathematics programs 
in mathematics, operating, and total costs and in overall cost effectiveness. 
The program ranked in the lowest quarter in mathematics achievement and 
operating costs. In overall rank order (weighted 40 percent achievement, 

40 percent costs, 20 percent program), the San Francisco mathematics 
program placed last, ranking thirteenth out of 13 projects. 




Long Beach Unified (reading). The Long Beach reading program rated 
high in program effectiveness, but relatively low in reading achievement 
(rank: 10.5 out of 15). The program ranked highest of 15 programs in 
reading costs, second highest in research and development costs, second 
highest in operating costs, and third highest in total expenditures per 
student. The program was the least cost effective of all 15 reading programs 
in reading, operating, and total costs and in overall cost effectiveness. 

The program ranked in the lowest quarter in operating costs per student. 

In overall rank order (weighted 40 percent achievement, 40 percent costs, 

20 percent program), the Long Beach reading program ranked twelfth out 
of 15 projects. 
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Evaluation of Demonstration Projects 



Projects were continually reviewed by the Division of Compensatory 
Education regarding their effectiveness in improving the achievement level 
of students in reading and mathematics. Through reports, questionnaires, 
observations, and interviews, projects were evaluated on the basis of 
several criteria involving program development, student achievement, and 
cost analysis. 

Program development criteria included the extent to which the projects 
adhered to the intent of the legislation and State Department of Education 
guidelines on eligibility, selection of participants, waivers, project organi- 
zation and administration, program content, demonstration concepts, staff 
development, and dissemination of information. 

Student achievement criteria included the extent to which the projects 
met objectives, measured results, and demonstrated effectiveness in 
improving student achievement levels. 

Cost analysis criteria included the extent to which the projects accounted 
for component costs and demonstrated cost effectiveness. 



Program Development Criteria 

Three separate ratings on program development criteria were combined 
to give a composite rank order of projects on program effectiveness. (See 
Table 1.*) 

First, projects were rated on a 14- point program visitation inventory, 
assessing (1) cost-effective elements implemented in grade seven; (2) cost- 
effective elements implemented in grade eight; (3) proportion of students 
served by the program; (4) demonstration opportunities; (5) performance 
objectives; (6) grouping practices; (7) planning; (8) inservice education; (9) 
dissemination of information; (10) individualiization of instruction; (11) use 
of material and personnel; (12) innovative arad exemplary program content; 
and (13) staff support. Projects were compared and ranked according to the 
ratings they received on this inventory. Table 1 reveals that eight projects 
tied for highest place on the program visitation inventory: S?-,^ita Safbara, 

El Monte, Colton, Riverside, Long Beach (mathematics). Long Beach (reading), 
San Francisco, and San Diego. Projects rated lowest were Stockton aiad 
Pasadena. Fresno, Los Angeles (Pacoima), Los Angeles (Belvedere), 

Los Angeles (Edison), and San Jose were also rated low. 



*A11 tables referred to in this publication will be found in Appendix A. 
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Projects were rated on a 78-item evaluation questionnaire, assessing the 
extent to which they adhered to guidelines on (1) eligibility; (2) selection of 
participants; (3) waivers; (4) program organization; (5) project administration; 

(6) program content; (7) demonstration activities; (8) staff development; (9) 
dissemination of information; (10) component costs; (11) evaluation procedures; 
and (12) research design. Projects were compared and ranked according to 
point scores on this questionnaire. As indicated in Table 1, Santa Barbara, 

El Monte, Colton, and Long Beach (mathematics) ranked highest on question- 
naire ratings; Los Angeles (Belvedere), Fresno, Los Angeles (Pacoima), and 
Los Angeles (Edison) were rated lowest. 

Projects were also rated on the basis of subjective observation notes on 
project operations, including interview data compiled during review team 
visits to project schools in the spring of 1971. Projects were compared and 
ranked according to observer ratings based on the general impressions 
obtained during these visits. Table 1 shows that highest observation ratings 
were given to the projects in Riverside, Santa Barbara, Montebello, and 
El Monte; lowest ratings were assigned to Pasadena, Fresno, Los Angeles 
(Pacoima), and Los Angeles (Belvedere). 

These three separate ratings on program development were combined into 
one program effectiveness rating by determining each project's median rank 
order. Projects were then ranked according to this single overall rating. 

Table 1 summarizes these ratings, which placed the following projects highest 
according to the overall rating on program effectiveness: Santa Barbara, 

El Monte, Colton, Riverside, and Long Beach (mathematics). Projects rated 
lowest in program effectiveness were Fresno, Pasadena, Los Angeles (Pacoim.a), 
and Los Angeles (Belvedere). 

On the informal questionnaire based on AB 938 guidelines, 38 items dis- 
tinguished those projects rated highest from those projects rated lowest. The 
pattern of responses to those questions gave a rough composite profile of the 
factors that tended to differentiate the most effecti’^^e from the least effective 
demonstration programs. 
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Profile of the Most Effective Projects 

In the most effective projects, the programs were being operated in one 
school in the district. The educational needs of the students and the assessment 
of those needs were described in detail. Project data showed very clearly how 
the program met the students' needs. It was evident that the project emphasized 
creativity. The project proposal was excellent and clearly explained the use of 
existing facilities. Over 75 percent of the students in the appropriate grade were 
included in the program. The most general grouping practice used was that of 
random order or heterogeneous groups. Instruction was mostly individualized. 

The project director was located at the school. Most of the project planning was 
done by the whole program staff working with resource personnel. Planning i 

included inservice training, a cooperative effort with a business or college, and : 

collection of curricular materials. 

i 

f 

? 

§ i 

i 

j 



r 



A. 



The most effective projects were clearly innovative and exemplary. Their 
methods, techniques, and procedures were described in detail, and were clearly 
related to the program objectives. Evidence was usually given of the competence 
of any cooperating agency involved. The impact of the demonstration aspects 
of the programs was such that they were seen as exemplary by other districts, 
and opportunities for observation were well planned and built in. Inservice 
education programs were intensive and effective, continuous, and mandatory. 

The inservice programs were clearly related to the objectives of the projects, 
with schedules and calendars described in the proposals. Each staff was con- 
sidered .outstanding, and each program was innovative and unique. Provisions 
were made for dissemination of information through publications, observations, 
and other methods. 

It was clearly demonstrated at the close of the project that the successful 
districts could maintain their programs, and strong intent was shown to do so. 

The most effective projects showed a level of cost effectiveness that made 
it possible for other, similar school districts to adapt these projects to their 
needs. The average cost per student in the successful projects during the first 
year was from $250 to $500, and the average increase in achievement was from 
11 to 15 months. Therefore, the calculated cost per student per month of growth 
was less than $50. The relationships between program, evaluation, project 
objectives, and expenditures were clearly indicated. The programs were very 
effective in improving the achievement level of students, with a degree of cost 
effectiveness best described as excellent. 



Profile of the Least Effective Projects 

In the least effective projects, the programs were being operated in more 
than one school in the district. The educational needs of the students and the 
assessment of those needs were poorly described. Project data showed fairly 
well how the program met these student needs. It was not evident that the 
unsuccessful projects emphasized creativity. The project proposals contained 
poor explanations of the use of existing facilities. From 25 to 75 percent of 
the students in the appropriate grade were included in the programs. The most 
general grouping practice used was that of homogeneous groups. Instruction 
was only slightly individualized. The project directors were not located at the 
participating schools. Most of the project planning was done by administrators 
not included in the program staff. Planning often omitted inservice training, a 
cooperative effort with a business or college, and collection of curricular 
materials. 

The least effective projects were possibly innovative and exemplary. 

However, their methods, techniques, and procedures were poorly described 
and were only vaguely related to the program objectives. Evidence of the 
competence of any cooperating agency involved was usually lacking. The impact 
of the demonstration aspects of the program was such that the participating 
schools were visited by personnel from other schools in their respective districts 
but not by persons from other districts. Opportunities for observation were very 
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poor. The inservice education programs in the least effective projects were 
fair but perfunctory and neither continuous nor mandatory. The inservice 
programs were somewhat related to the objectives of the projects, but no 
schedules or calendars were described in the proposals. The staffs of the 
unsuccessful projects were considered fair or good. There v/as scant pro- 
vision for the dissemination of inform^ation through publications, observations, 
or other methods. 

In the least effective projects, comparison groups either were not used or 
were not described. The least effective projects were moderately successful 
during their first year but showed a level of cost effectiveness that made it 
difficult for other, similar school districts to adapt the projects to their needs. 
It was often uncertain whether the unsuccessful districts could maintain their 
programs at the close of the project, and only slight intent was show^p do so. 

The average cost per student in the least effective projects during -the first 
year was from $501 to $750, and the average increase in achievement was from 
six to ten months. Therefore, the calculated cost per stude/?^ per month of— 
growth was more than $50. The relationships between progrtim, evaluation, 
project objectives, and expenditures were vaguely indicated. The programs 
were barely effective in improving the achievement level of students, with a 
degree of cost effectiveness best described as poor. 



Student Achievement 

Standardized test results, as reported for each project, were tabulated 
and compared to show increases in achievement in reading and mathematics. 
Achievement increases were expressed in terms of months of gain in mean 
grade placement for each month of instruction. Achievement was measured 
by the Comprehensive Tests of Basic Skills , Level 3, forms Q or R. This 
was administered as the pretest and post- test in all projects. Each project 
was rated separately for reading and mathematics. 

As shown in Table 2, projects rated highest in mean achievement gains in 
reading were Colton, Oakland, Santa Barbara, and Los Angeles (Edison). 
Projects reporting lowest reading gains were Stockton, San Diego, Los Angeles 
(Belvedere), and Montebello. 

In mathematics, the projects rated highest in mean achievement gains were 
Los Angeles (Pacoima), Colton, Pasadena, and Long Beach. The mathematics 
project showing the lowest achievement gain was San Francisco, followed by 
Stockton, Los Angeles (Belvedere), and Riverside. 

To compare projects in which the number of months of instruction varied, a 
ten-month school year was considered the basis for determining instructional 
gain. Each monthly gain ratio reported was therefore converted to a yearly 
gain figure; for example, Colton's reported gain of 1.9/1 (1.9 months' gain per 
month) was converted to 19 months' gain for the school year. Similar figures 
were derived by multiplying each gain ratio in Table 2 by tan. 
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Cost Analysis 

Cost-analysis ratings were compiled for per- student expenditures and 
cost effectiveness. Cost effectiveness was defined as the greatest increase 
in student achievement for the least cost per student. Projects were rated 
separately on reading and mathematics, with the least cost receiving the 
highest rating. 



Per -Student Expenditures 

Expenditures were summarized as reported. Costs were tabulated as 
of April 30, 1971, for the separate categories of (1) reading instruction; 

(2) mathematics ins5truction; (3) design and implementation (reported as 
"research and development"); (4) operating expenses; and (5) total expendi- 
tures per studen’t. Projects were compared and ranked in order in each 
category of expenditures. These rankings are presented in tables 3 through 7. 

Table 3 reveals that the projects with the least expenditures per student for 
reading instruction were Colton, Montebello, Los Angeles (Belvedere), and 
Los Angeles (Pacoima). Projects With the greatest expenditures were Long ' 

Beach, El Monte, Oakland, and Santa Barbara. The per- student expenditure 
in the "most expensive" project was 11 times that of the "least expensive" | 

project. I 

i 

Table 4 shows that projects with the least expenditures per student for 
mathematics instruction were again Colton, Montebello, Los Angeles (Belvedere), 
and Los Angeles (Pacoima). Projects reporting the greatest expenditures were | 

Long Beach, Oakland, San Jose, and San Francisco. The average per-student { 

expenditure in the "most expensive" project amounted to ten times that of the | 

"least expensive" project. j 

1 

] 

Design and ijnplementation costs ranged from $3 to $508 per student. j 

Table 5 indicates that these expenditures were lowest in the Stockton, Pasadena, j 

San Francisco, and Colton projects. They were highest in the San Jose, Long | 

Beach, and San Diego projects. ! 

Table 6 shows that the projects reporting the lowest per-student operating 
expenses were Colton and Montebello, followed by Riverside and Los Angeles i 

(Belvedere). The highest operating expenses were recorded for Oakland, Long 1 

Beach (reading), San Francisco, and El Monte. Operating expenses for Oakland 1 

were 9.4 times those for Colton and Montebello. i 

i 

As shown in Table 7, total expenditures per student were lowest in Colton, | 

Montebello, Pasadena, and Los Angeles (Belvedere). Greatest expenditures i 

were reported for Oakland, San Jose, and Long Beach. Total expenses ranged 
from $145 per student in the "least expensive" project to $1, 320 per student 
in the "most expensive" project. 




9 

U 



Cost Effectiveness 



Increase in achievement per student was compared with expenditures per 
student to give a measure of cost effectiveness, as operationally defined. 

Cost effectiveness was expressed in two ways: (1) the number of months of 
achievement attained for each dollar spent per student; and (2) the cost in 
dollars per student of each month of achievement attained. 

The cost effectiveness of projects in terms of per- student expenditures 
for reading instruction appear in Table 8 and for mathematics instruction 
in Table 9. Cost effectiveness data for design and implementation are given 
in tables 10 and 11, for operating expenses in tables 12 and 13, and for total ’ ! 

expenditures in tables 14 and 15. i 

Table 8 reveals that the cost effectiveness of reading instruction ranged ; 

from one month of achievement for $3. 74 per student (Colton) to one month ( 

of achievement for $79 (Long Beach). Table 9 shows that the cost effective- I 

ness of mathematics instruction ranged from Colton's $3.95 per student for 
each month of achievement to San Francisco's $84.25 per student. \ 

Table 10 shows that the cost effectiveness of design and implementation 
in reading projects ranged from 38 cents per student for each month of reading 1 

achievement (Stockton) to $39.08 per student (San Jose) Table 11 shows that i 

the cost effectiveness of design and implementation in mathematics projects j 

ranged from 33 cents per student for each month of mathematics achievement ^ 

(Stockton) to $36. 29 per student (San Jose). 

j 

As indicated in Table 12, the cost effectiveness of operating expenses in 
reading projects ranged from one month of reading achievement for $6.89 
per student (Colton) to one month of achievement for each $79 (Long Beach). -I 

The cost effectiveness of operating expenses in mathematics projects,as ’ 

shown in Table 13, ranged from Colton's $6.89 per student to San Francisco's i 

$163. 25 per student for each month of mathematics achievement. i 

Table 14 reveals that the cost effectiveness of total expenditures in reading 
projects ranged from one month of reading achievement for each $7.63 spent 
in Colton to one month for each $79 spent in Long Beach. Comparable figures ; 

for mathematics projects, as shown in Table 15, ranged from a cost of $7.63 
per student in Colton to $165.75 per student in San Francisco. ; 

After the projects were compared and ranked in order on the four cost- ,• 

effectiveness measures, the median rank order and the composite rank of the 
projects were calculated. The median rank order assigned was the median of f 

the ranking of each project on the four measures listed. The composite rank j, 

was the relative rank order of projects on median rank orders. The composite 1 
rank order of the projects on cost effectiveness is given in Table 16 and Table 17. | 

Table 16 shows that reading projects rated highest with respect to cost effective- | 
ness were Colton, Montebello, Los Angeles (Pacoima), and Los Angeles (Edison). .1 
Reading projects rated as least cost effective included Long Beach, Oakland, ] 

San Jose, and Fresno. ^ 






The rank order of 'mathematics projects oncost effectiveness, given in 
Table 17, shows that the most cost-effective projects were Colton, Los 
Angeles (Pacoima), Montebello, and Pasadena.' Least cost effective were 
San Francisco, Oakland, San Jose, Stockton, and Long Beach. 



Combined Rank Order Rating of Projects 

Following the suggestions of the State Advisory Committee on Program and 
Cost Effectiveness, projects were rated on overall effectiveness, combining 
their rank- order ratings on three factors weighted as follows: achievement 
gains, 40 percent; operating costs per student, 40 percent; and program effective- 
ness, 20 percent. The rankings t|jat were combined to give the rating on overall 
effectiveness are presented in tables 18 and 19. 

Table 18 indicates that the reading projects rated highest in overall effective- 
ness were Colton, Santa Barbara, Riverside, and Montebello; projects rated 
lowest were Los Angeles (Belvedere) and Fresno, then Stockton, Long Beach, 
and San Diego. 

Mathematics projects ranked highest in overall effectiveness, as shown in 
Table 19, included Colton, Riverside, Long Beach, and Montebello; projects 
rated lowest were San Francisco, Fresno, Los Angeles (Belvedere), and 
Stockton. 

It was agreed that no project could be considered cost effective that was not 
first of all effective in improving achievement. The criterion for effectiveness 
in achievement gains was arbitrarily set at the median gain for all projects, 
as reported. It was also agreed that projects falling in the lowest quarter on 
several separate ratings should be considered least effective. Quartile rankings 
of projects are summarized in Table 20. 

It was further agr*eed that program effectiveness should be considered . 
separately from cost effectiveness in determining the least effective projects. 
Projects were ranked in order of overall effectiveness and in order of cost 
effectiveness, as defined. 





Evaluation Results 



Evaluation results reveal that the projects rated highest on program 
effectiveness were Santa Barbara, El Monte, Colton, Riverside, and Long 
Beach (mathematics). Projects rated lowest in program effectiveness 
included Fresno, Pasadena, Los Angeles (Pacoima), and Los Angeles 
(Belvedere). & 

Reading projects rated highest on student achievement in reading were 
Colton, Oakland, S^nta Barbara, and Los Angeles (Edison). Projects 
rated lowest on acl^evement in reading were Stockton, San Diego, Los 
Angeles (Belve dere ), and Montebello. 

f ' 

Mathematics projects reporting highest student achievement included 
Los Angeles (Pacoima), Colton, Pasadena, and Long Beach. Mathematics 
projects rated lowest in achievement were San Francisco, Stockton, Los 
Angeles (Belvedere), and Riverside. 

Projects reporting lowest operating expenses per student were Colton, 
Montebello, Riverside, and Los Angeles (Belvedere). Highest per-student 
operating costs were recorded for Oakland, Long Beach (reading), San 
Francisco, and El Monte. 

Lowest total expenditures per student were reported by Colton, Montebello, 
Pasadena, and Los Angeles (Belvedere). Highest expenditures per student 
were for Oakland, San Jose, and Long Beach. 

Reading projects rated highest on cost effectiveness were Colton, Montebello, 
Los Angeles (Pacoima), and Los Angeles (Edison). Reading projects rated as 
least cost effective included Long Beach, Oakland, San Jose, and Fresno. 

Mathematics projects rated highest on cost effectiveness included Colton, 

Los Angeles (Pacoima), Montebello, and Pasadena. The least cost-effective 
mathematics projects were San Francisco, Oakland, San Jose, Stockton, and 
Long Beach. 

Reading projects rated highest in overall effectiveness were Colton, Santa 
Barbara, Riverside, and Montebello. Those rated lowest included Los Angeles 
(Belvedere), Fresno, Stockton, Long Beach, and San Diego. 

Mathematics projects rated highest in overall effectiveness included Colton, 
Riverside, Long Beach, and Montebello. Rated lowest in overall effectiveness 
were San Francisco, Fresno, Los Angeles (Belvedere), and Stockton. 

Projects ranking above the highest quartile most often were Colton and 
Riverside. Projects placing below the lowest quartile most often were San 
Francisco and Los Angeles (Belvedere). 



On the basis of these evaluations, it was recommended that (1) the most 
effective demonstration programs should be retained, extended, and com- 
mended; (2) effective programs should be retained, improved, and strengthened; 
and (3) the least effective programs should be terminated. 
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Appendix A 

Statistical Data 

TABLE 1 

RANK ORDER OF PROJECTS ON PROGRAM EFFECTIVETESS 
Spring, 1971 



Protect 


Senarate Rank>Order Ratines* 


Median 


Overall 


lU 


121 


131 


Ratine 


Rank Order 


Santa Barbara 

V 


4.5 

ii 


1 

' . * 


2 


2 


1 


El Monte 


4.5 


2 


4 


4 


2 


Colton 


4.5 


3 


8 


4.5 


4 


Riverside 


4.5 


8 


1 


4.5 


4 


Long Beach (M)** 


4.5 


4 


5 


4.5 


4 


Long Beach (R) 


4.5 


5 


6 


5 


6 


Montebello 




10 


3 


6.5 


7 


San Francisco 


4.5 


12 


9 


9 


8.5 


Oakland 


9 


9 


7 


9 


8.5 


San Jose 


12 


7 


10 


10 


10 


Stockton 


15.5 


6 


11 


11 


11.5 


San Diego 


4.5 


11 


12 


11 


11.5 


L.A. - Edison 


12 


14 


13 


13 


13 


L.A. " Belvedere 


12 


17 


14 


14 


14 


L.A. - Pacoina 


12 


15 


15 


15 


15 


Pasadena 


15.5 


13 


17 


15.3 


16. 


Fresno 


12 


16 


16 


16 

1 


17 



*Ratlng8: (1) Program visitation Inventory 

(2) Evaluation questionnaire 
p) Observation notes 



16 

15 



**M=mathematics ; R=reading 



TABLE 2 



RANK ORDER OF PROJECTS ON MEAN ACHIEVEMENT GAINS 

1970-71 



Readins Achievement (N"1S) 


Mathematics Achievement 


(N-13) 


Rank 


Proiect 


Gains* 


Rank 


Proiect 


Gains* 


1 


Colton 


1.9/1 


1 


L.A.-Pacoima 


2.1/1 


2 


Oakland 


1.7/1 


2 


Colton 


1.9/1 


3 


Santa Barbara 


1.6/1 


3 


Pasadena 


1.7/1 


4 


L. A. -Edison 


1.4/1 


4 


Long Beach (M)** 


1.6/1 


6.5 


Riverside 


1.3/1 


6 


Oakland 


1.4/1 


6.5 


San Jose 


1.3/1 


6 


San Jose 


1.4/i 


6.5 


L.A.-Pacoima 


1.3/1 


6 


Fresno 


1.4/1 


6.5 


San Francisco 


1.3/1 


8 


L.A. -Edison 


1.1/1 


9 


El Monte 


1.1/1 


9 


Montebello 


1.0/1 


10.5 


Fresno 


1.0/1 


i 

11 


Riverside 


0.9/1 


10.5 


Long Beach (R)^ 


1.0/1 


11 


L.A. -Belvedere 


0.9/1 


12.5 


Montebello 


0.9/1 


11 


Stockton 


0.9/1 


12.5 


L.A. -Belvedere 


0.9/1 


13 


San Francisco 


0.4/1 


14.5 


San Diego 


0.8/1 








14.5 


Stockton 


0.8/1 






- 




Median: 


1.3/1 




Median: 


1.4/1 



*Gains : 



Number of months of academic achievement gains reported per month of 
instruction. 





**M=ma thema tics 
t R=reading 



TABLE 3 



RANK ORDER OF PROJECTS OH EXPENDITURES PER STUDENT 
Reading Instruction 
(April 30, 1971) 



Rank 


Proiect 


Amount 


1 


Colton 

■# 


$ 71.00 


2 T 


Montebello 


86.00 


3 


Belvedere 


145.00 


4 


Pacolna 


147.00 


5 


Edison 


169.00 


6 


Riverside 


200.00 


7 


Stockton 


272.00 


8 


San Francisco 


325.00 


9 


Fresno 


381.00 


10 


San Diego 


464.00* 


11 


San Jose 


504.00 


12 


Santa Barbara 


534.00 


13 


Oakland 


660.00 


14 


El Monte 


735.00 


IS 


Long Beach 


790.00 





*Estlmated 



TABLE 4 



RANK ORDER OF PROJECTS ON EXPENDITURES PER STUDENT 
Mathematlcg Initructlon 
(April 30. 1971) 



Rank 


Proiect 


Amount 


1 


Colton 


$ 75.00 


2 


Montebello 


89.00 


3 


Belvedere 


145.00 


4 


Pacoima 


147.00 


5 


Edlaon 


169.00 


6 


Riveraide 


206.00 


7 


Paaadena 


268.00 


8 


Stockton 


272.00 


9 


Fresno 


335.00 


10 


San Francisco 


337.00 


11 


San Jose 


504.00 


12 


Oakland 


660.00 


13 


Long Beach 


769.00 
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TABLE 5 



RANK ORDER OF PROJECTS ON EXPENDITURES PER STUDENT 
Design and Implementation 
(April 30, 1971) 



Rank 


District 


Program 


Amount 


1 


Stockton 


. R & M* 


$ 3.00, 


2 


* Pasadena 


M 


12.00 ' 


3.5 


San Francisco 


R & M 


15.00 


3.5 


Colton 


R & M 


15.00 


5 


Pacoima 


R & M 


27.00 


6 


Belvedere 


R & M 


40.00 


7 


Montebello 


R & M 


44.00 


8 


Edison 


R & M 


45.00 


9 


Oakland 


R & M 


89.00 


10 


Santa Barbara 


R 


98.00 


11 


El Monte 


R 


110.00 


12 


Fresno 


R & M 


116.00 


13 


Riverside 


R & M 


155.00 


14 


San Diego 


R 


176.00** 


15 


Long Beach 


M 


215.00 


16 


Long Beach 


R 


278.00 


17 


San Jose 


R & M 


508.00 



*R=reading; M*=matheraatics 
**Estimated 
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TABLE 6 



RANK ORDER OF PROJECTS ON EEPENDITURES PER STUDENT 
Operating Expcnics 
(April 30, 1971) 



Rank 


Diatrict 


Program 


AsM>unt 


1.5 


Colton 


RAM* 


$ 131.00 


1.5 


Montebello 


R & M 


131.00 


3 


Rivera Ide 


R & M 


251.00 


4 


Belvedere 


R & M 


252.00 


5 


Pasadena 


M 


257.00 


6 


Pacoima 


R & M 


267.00 


7 


San Diego 


R 


288.00* 


8 


Edison 


R & M 


293.00 


9 


Santa Barbara 


R 


436.00 


10 


Long Beach 


M 


486.00 


11 


San Jose 


R & M 


500.00 


12 


Stockton 


R & M 


541.00 


13 


Fresno 


R & M 


599.00 


14 


El Monte 


R 


626.00 


15 


San Francisco 


R & M 


653.00 


16 


Long Beach 


R 


790.00 


17 


Oakland 


R & M 


1,231.00 



*R=readlng; M=mathematics 
**Estlmated 




TABLE 7 



RAmC ORDER OF PROJECTS ON TOTAL EXPENDITDRES PER STUDENT 
(April 30. 1971) 



Rank 


District 


Proeraa 


AsKMint 


1 * 


Colton 


R & M* 


$ 145.00 


2 


Montebello 


R & M 


175.00 


3 


Pasadena 


M 


268.00 


4 


Belvedere 


R & M 


291.00 


5 


Pacoima 


R & M 


294.00 


6 


Edison 


R & M 


338.00 


7 


San Diego 


R 


465.00 ** 


8 


Riverside 


R & M 


486.00 


9 


Santa Barbara 


R 


534.00 


10 


Stockton 


R & M 


543.00 


11 


San Francisco 


R & M 


663.00 t 


12 


Fresno 


R & M 


716.00 


13 


El Monte 


R 


735.00 


14 


Long Beach 


M 


769.00 t 


15 


Long Beach 


R 


790.00 t 


16 


San Jose 


R & M 


1,008.00 


17 


Oakland 


R & M 


1,320*00 



*R=reading; M=mathematics 

**Ettimated 

t Includes snendments 6/10/71: San Francicco— $12 ,440 



Long Beach—* $15. 735 




imE 8 



COST EFEECTIVEMESS OF READING PROJECTS 

IN TERMS OF PER- STUDENT EXPENDITURES FOR 

READING INSTRUCTION 



Rank 


Proiect 


Nuaber of Months of 
AchievesMnt in Reading 
Fbr Each Dollar Spent 


Cost in Dollars Per Student 
of Each Month of 
Achieveaent in Reading 

IL. 


I 


Colton 


0.27 , 


$ 3. .74 


2 


Montebello 


0.10 


9-36 


3 


L.A. - Pacoima 


0.09 


11.31 


4 


L.A. - Edison 


0.08 


12.07 


5 


Riverside 


0.07 


15.38 


6 


L.A. - Belvedere 


0.06 


16.11 


7 


San Francisco 


0.04 


25.00 


8 


Santa Barbara 


0.03 


33.38 


9 


Stockton 


0.03 


34.00 


10 


Fresno 


0.03 


38.10 


II 


San Jose 


0.03 


38.77 


12 


Oakland 


' 0.03 


38.82 


13 


San Diego 


0.02 


58.00 


14 


El Monte 


O.OI 


66.82 


15 


Long Beach 


O.OI 


79.00 



( 




22 




TABLE 9 

COST EFFECTIVENESS OF MATH PROJECTS 
IN TERMS OF PER-STUDENT EXPENDITURES FOR 
MATHEMATICS INSTRUCTION 



Rank, 


Proiect 


Kunber of Months of 
Achievement in Math for 
Each Dollar Spent ' 


Cost in Dollars Per Student 
of Each Month of 
Achievement in Math 


1 


Colton 

9 


0.25 


$ 3.95. 


2 


L.A. - Pacoixna 


0.14 


7.00 


3 


Montebello 


0.11 


8.90 


4 


L.A. “ Edison 


0.07 


15.36 


5 


Pasadena 


0,06 


15.76 


6 


L.A. - Belvedere 


0.06 


16.11 


7 


Riverside 


0.04 


22.89 


8 


Fresno 


0.04 


23.93 


9 


Stockton 


0.03 


30.22 


10 


San Jose 


0.03 


36.00 


11 


Oakland 


0.02 


47.14 


12 


Long Beach 


0.02 


48.06 


13 


San Francisco 


0.01 


84.25 
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TABLE 10 



COST EFFECTIVENESS OF READING PROJECTS 
IN TERMS OF PER-STUDENT EXPENDITURES FOR 
DESIGN AND IMaEMENTATION 



Rank 


Project 


Number of Months of 
Achievement In Reading 
For Each Dollar Soent 


Cost In Dollars Per Student 
of Each Month of 
Reading Achievement 


1 


Stockton 


2.67 


$ 0.38 


2 


Colton r 

I 


0 

1.27 


0.79 


3 


San Francisco 


0.87 


1.15 


4 


L.A. - Pacoinia 


0.48 


2.08 


5 


L.A. <• Edison 


0.31 


• 3.21 


6 


L.A. - Belvedere 


0.23 


4.44 


7 


Montebello 


0.20 


4.89 


8 


Oakland 


0.19 


5.24 


9 


Santa Barbara 


0.16 


6.13 


10 


El Monte 


0.10 


10.00 


11 


Fresno 


0.09 


11.60 


12 


Riverside 


0.08 


11.92 


13 


San Diego 


0.05 


22.00 


14 


Long Beach 


0.04 


27.80 


15 


San Jose 


0.03 


39.08 



-I 
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TABLE 11 



COST EFFECTIVENESS OF MATH PROJECTS 
IN TERMS OF PER-STUDENT EXPENDITURES FOR 
DESIGN AND IMPLEMENTATION 



Rank 


Proiect 


Number of Months of 
Achievement in Math 
for Each Dollar Spent 


Cost in Dollars Per Student 
of Each Month of 
Achievement in Math 


1 


Stockton 


3.00 


$ 0.33 


2 


Pasadena 


1.42 


0.71 


3 


Colton 


1.27 


0.79 


4 


L.A. - Pacoima 


0.78 


1.29 


5 


San Francisco 


0.27 


3.75 


6 


L.A. - Edison 


0.24 


4.09 


7 


Montebello 


0.23 


4.40 


8 


L.A. - Belvedere 


0.23 


4.44 


9 


Oakland 


0.16 


6.36 


10 


Fresno 


0.12 


8.29 


11 


Long Beach 


0.07 


13.43 


12 


Riverside 


0.06 


17.22 


13 


San Jose 


0.03 


36.29 





TABLE 12 



COST EFFECTIVENESS OF READING PROJECTS 
IN TERMS OF PER-STUDENT EXPENDITURES FOR 
OPERATING EXPENSES 



Rank 


Pro.iect 


Number of Months of 
Reading Achievement 
for Each Dollar Spent 


Cost in Dollars per Studen 
of Each Month of 
Reading Achievement 


1 


Colton 


0.15 


$ 6.89 


2 


Montebello 


0.07 


14.56 


3 


Riverside 


0.05 


19.31 


A 


L.A. - Pacoima 


0.05 


20.54 


5 


L.A. - Edison 


0.05 


20.93 


6 


Santa Barbara 


0.04 


27.25 


7 


L.A. - Belvedere 


0.04 


28.00 


8 


San Diego 


0.03 


36.00 


9 


San Jose 


0.03 


38.46 


10 


San Francisco 


0.02 


50.23 


11 


El Monte 


0.02 


56.91 


12 


Fresno 


0.02 


59.90 


13 


Stockton 


0.01 


67.63 


lA 


Oakland 


0.01 


72.41 


15 


Long Beach 


0.01 


79.00 



4 

I 

I 

i 

) 
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TABLE 13 



COST EFFECTIVENESS OF MATH PROJECTS 
IN TERMS OF PER- STUDENT EXPENDITURES FOR 
OPERATING EXPENSES 



Rank 


Proiect 


Number of Months of 
Math Achievement 
for Each Dollar Spent 


Cost in Dollars Per Studei 
of Each Month 
of Math Achievement 


1 


Colton 


0.15 


$ 6.89 


2 


L.A. - Pacoima 


0.08 


12,71 


3 


1* 

Montebello 


0.08 


13.10 


4 


Pasadena 


0.07 


15.12 


5 


L.A. - Edison 


0.04 


26.64 


f 

6 


Riverside 


0.04 


27.89 


7 


L.A. - Belvedere 


0.04 


28.00 


8 


Long Beach 


0.03 


30.38 


9 


San Jose 


0.03 


35.71 


10 


Fresno 


0.02 


42.79 


11 


Stockton 


0.02 


60.11 


12 


Oakland 


0.01 


87.93 


13 


San Francisco 


0.01 


163.25 




27 




TABLE 14 



COST EFFECTIVENESS OF READING PROJECTS 
IN TERMS OF PER-STUDENT 
TOTAL EXPENDITURES 



Rank 


Prolect 


Number of Months of 
Reading Achievement 
for Each Dollar Spent 


Cost In Dollars Per Student 
of Each Month of 
Achievement In Reading 


1 


Colton 


0.13 


$ 7.63 


2 


Montebello 


.0.05 


19.44 . 


3 


L.Ai - Facolma 


0.04 


22.62 


4 


L.A. - Edison 


0.04 


24.14 


5 


Li A. - Belvedere 


0.03 


32.33 


6 


Santa Barbara 


0.03 


33.38 


7 


Riverside 


0.03 


. 37.38 


8 


San Francisco 


0.02 


51.00 


9 


San Diego 


0.02 


58.13 


10 


El Monte 


0.01 


66.82 


11 


Stockton 


0.01 


67.88 


12 


Fresno 


0.01 


71.60 


13 


San Jose 


0.01 


77.54 


14 


Oakland 


0.01 


77.65 


15 


Long Beach 


0.01 


79.00 



I 



O 
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TABLE 15 



COST EFFECTIVENESS OF MATH PROJECTS 
IN TERMS OF PER- STUDENT 
TOTAL EXPENDITURES 



Rank 


Project 


Number of Months of 
Math Achievement 
for Each Dollar Spent 


. Cost in Dollars Per Student 
of Each Month of 
Achievement in Math 


1 


Col ton 


0.13 


$ 7.63 


2 


,L.A. - Pacoitna 


. 0.07 


lA.OO 


3 


Pasadena 


0.06 


15.76 


A 


Montebello 


0.06 


17.50 


5 


L.A. - Edison 


0.03 


30.73 


6 


L.A. - Belvedere 


0.03 


32.33 


7 


Long Beach 


0.02 


A8.00 


8 


Fresno 


0.02 


51.14 


9 


Riverside 


0.02 


54.00 


10 


Stockton 


0.02 


60.33 


11 


San Jose 


0.01 


72.00 


12 


Oakland 


0.01 


94.29 


13 


San Francisco 


0.01 


165.75 



ERIC 
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TABLE 16 



RANK ORDER OF READING PROJECTS ON COST EFFECTIVENESS 





Four Cost-Effectiveness Measures 


Median 
Rank 
Order on 
Four C/E 
MsASures 


Composite 

Rank 

(Relative 

Rank 

Order) 


Project. 


Reading 

Costa 


Research 
& . 

Developnent 


Operating 

Coats 


Total 

Costs 
















Colton 


1 


V 

2 


1 


1 


1 


1 


Montebello 


2 


7 


2 


2 


2 


2 


L.A.-Pacolma 


3 


4 


4 


3 


3.5 


3 


L. A. <• Edison 


4 


5 


5 


4 


4.5 


4 


L. A. -Belvedere 


6 


6 


7 


5 


6 


5.5 


Riverside 


5 


12 


3 


7 


6 


5.5 


Sants Barbara 


8 


9 


6 


6 


7 


7 


San Francisco 


7 


3 


10 


8 


7.5 


8 


Stockton 


9 


1 


13 


11 


10 


9 


El Monte 


14 


10 


11 


10 


10.5 


10 


San Diego 


13 


13 


8 


9 


11 


11 


. Fresno 


10 


11 


12 


12 


11.5 


12 


San Jose 


11 


15 


9 


13 


12 


13 


Oakland 


12 


8 


14 


14 


13 


14 


Long Beach 


15 


14 


15 


15 


15 


1 
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TABLE 17 



RANK OBi)ER OF MATHEMATICS PROJECTS OH COST EFFECTIVEHESS 



Proiect 


Four Cost-Effectiveness Measures 


Median 
Rank 
Order on 
Four C/E 
Measures 


Composite 

Rank 

(Relative 

Rank 

Order) 


Math 

Costs 


Research 

6 

Development 


Operating 

Costs 


Total 

Costs 












Colton 


1 


3 


1 


1 


1 


1 


L.A.-Pacoima 


2 


4 


2 


2 


2 


2 


Montebello 


3 


7 


3 


4 


3,5 


3.5 


Pasadena 


5 


2 


4 


3 


3.5 


3.5 


L. A. -Edison 


4 


6 


5 


5 


5 


5 


L. A. -Belvedere 


6 


8 


7 


6 


6.5 


6 


Riverside 


7 


12 


6 


9 


8 


7 


Fresno 


8 


10 


10 


8 


9 


8 


Long Beach 


12 


11 


8 


7 


9.5 


9.5 


Stockton 


9 


1 


11 


10 


9.5 


9.5 


San Jose 


10 


13 


9 


11 


10.5 


11 


Oakland 


11 


9 


12 


12 


11.5 


12 


San Francisco 


13 


5 


13 


13 


13 


13 



O 
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lAlLE 18 



BANK ORDER RATING OF READING PROJECTS ON 
OVERALL EFFECTIVENESS 

June, 1971 





Rank Order 


on Contributing Factors 


Rank Order 
On 

Overall 

Effectiveness 


15 Frolects 


Achievement 

Gains 

(Weighted 40X) 

j 


Program 
Effectiveness 
(Weighted 201) 


Operating 

Costs 

(Weighted 40%) 


Colton 


1 


3.5 


1.5 


1 


Santa Barbara 


3 


1 


8 


2 


Riverside 


6.5 


3.5 


3 


3 


Montebello 


5 


6 


1.5 


4 


Los Angeles-Facolma 


6.5 


14 


5 


5 


Los Angeles'Edison 


4 


12 


7 


6 


Oakland 


2 


7.5 


15 


7.5 


San Francisco 


6.5 


7.5 


13 


7.5, 


El Monte 


9 


2 


12 


9.5 


San Jose 


6.5 


9 


9 


9.5 


San Diego 


14.5 


10.5 


6 


12 


Long Beach 


10.5 


5 


14 


12 


Stockton 


14.5 


10.5 


10 


12 


Fresno 


10.5 


15 


11 


14 


Los Angeles 'Belvedere 


12.5 


13 


4 


15 
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1ABLE 19 



RANK ORDER RATING OF MATHEHATICS PROJECTS ON 
OVERALL EFFECTIVENESS 

June, 1971 



13 Prelects 


Rank Order 


on Contributing Factors 


Rank Order 
On 

Overall 

Effectiveness 


Achievement 

Gains 

(Weighted 40%) 


Program 
Effectiveness 
(Weighted 20%) 
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II 
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II 


10 
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10.5 
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II 


12 


San Frr.ncisco 


13 
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12 


13 
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TABLE 20 . * 

qdartile ranking of projects on achievement 

GAINS, PROGRAM EFFECTIVENESS, AND OPERATING COSTS 

June, 1971 
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High Achievement Gains 
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Next to 
Higheat 

Quarter 
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Quarter 
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Colton 

Santa Barbara 

L.A.-Ediaon 

Oakland 



Riverside 
San Francisco 
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San Joae 



El Monte 
Freano 
Long Beach 



Montebello 
L.A.'Belvetiiers 
Stockton 
San Diego 



Math 

(N-13) 



Hig^ Program 
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(N-17) 



Colton flColton ^ 

L.A.'Pacoima II Santa Barbara 
Pasadena El Monte 

Riverside 
Long Beach ** 



Fresno 
San Jose 
Oakland 
Long Beach 



Montebello 
L. A. -Edison 



Montebello 
San Francisco 
Oak lend 
Lok 5 Beach 



Stockton 
San Diego 
L.A.-Ediaon 
San Jose 



L.A.-Belvedere||L. A. -Belvedere 
Stockton RL.A.-Pacoima 

Riverside H Pasadena 

San Francisco fl Fresno 



Low Operating Costs Per Student 



Reading 

(N-15) 



Colton 
Montebello 
Riverside 
L. A. -Belvedere 



L.A.-Pacoina 
San Diego 
L.A. -Edison 
Santa Barbara 



San Jose 
Stockton 
Fresno 
El Monte 



San Francisco 
Long Beach 
Oakland 



Math 

(N-13) 



Colton . 
Montebello' 
L.A. -Belvedere 
Riverside 



Pasadena 
L.A.-Pacoima 
L.A .-Edison 



Long Beach 
San Jose 
Stockton 



Fresno 

San Francisco 
Oakland 



*Ad Justed to allow for tied rank orders. 

**M=mathemat ic s 
t R=feading 
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Appendix B 

A Summary of the Most Effective Project: 

Demonstration Programj in Reading and Mathematics 
at Colton Junior High School 

As documented in their 1970-71 annual report, the demonstration programs 
in reading and mathematics at Colton Junior High School were highly success- 
ful.^ The following is a brief summary of the planning, program, and evalua- 
tion activities that resulted in the highest level of student performance and 
cost effectiveness of any demonstration project in California during 1970-71. 



Project Planning at Colton 

Colton Junior High School's demonstration project was planned in December, 
1969, and implemented during the 1970 spring semester. It was designed 
primarily as a three-year spiraling program, with student skills built one upon 
the other to achieve the following goals : (1) allow students to assume more of 
the responsibili'ty for their own education; (2) improve academic achievement in 
all subjects by improving reading and mathematics skills; and (3) -'mprove 
student attitudes. 

The original project proposal was written by a team of teachers, adminis- 
trators, and school district personnel. This team approach has been continued 
through each year of the project's operation. There were two instructiorial 
teams, one composed of four reading teachers and one composed of four math 
teachers. These two groups scheduled, planned, and evaluated activities together. 
The project director scheduled approximately 120 students for each period of 
mathematics and each period of reading; then the teams assigned students to 
individual classes. 

Three basic types of learning arrangements were used extensively in the 
Colton project: large group instruction, small group discussion, and inde- 
pendent study. 



Program Innovations in the Colton Project 

Since the reading program was ungraded and utilized miniunits, students 
were assigned to individual classes on the basis of interest and diagnosed needs. 



^"AB-9 38 Demonstration Program, Second Annual Report." Colton, Calif.: 
Colton Joint Unified School District, June 10, 1971 (mimeographed). 




35 




Miniunits were offered at five- or six-week intervals, and both students and 
teachers were allowed to sign up to study (or teach) the topics that appealed 
to them. Some of the units included during 1970-71 v/ere propaganda devices, 
the newspaper, detective fiction, good grooming, motorcycles, vocations, 
scriptwriting, and reference skills. Paperbacks, field trips, reading machines, 
filmstrips, speakers, movies, newspapers, and teacher-prepared materials 
were used to help individualize the units. 

The reading laboratory was used for a variety of instructional purposes, 
including remediation and teaching of the miniunits. The lab was equipped 
with controlled readers and accompanying filmstrips, listening centers, 
individual study carrels, tape recorders, a language master machine, and 
film equipment. Software items included programmed reading workbooks, 
paperbacks, Sullivan materials, selected stories, textbooks and brochures 
for individual study, and teacher-prepared pamphlets. 

The mathematics program was not ungraded; team members met after 
the diagnostic testing, however, and assigned students to the instructors who 
stressed the areas of student need indicated by the tests. This team also 
prepared its own booklets and teaching materials so that students could 
progress at their own speed and work on the units they particularly needed. 

The math booklets emphasized basic skills, so that students could enter the 
regular state-adopted mathematics program as soon as possible. 

Both teams met weekly for planning, inservice training, discussion, and 
evaluation. The project director, who was also the school principal, scheduled 
all teachers of a subject to the same conference period for these weekly 
meetings. 



Continuous Evaluation 

Outside evaluation was provided quarterly to give direction and recommenda- 
tions and to assess the attainment of goals and ol^jectives. Student and faculty 
interviews, questionnaires, observations, and test data were included, based 
upon weekly visits of the project evaluator. 

Both standardized and nonstandardized tests were used in the evaluation. 
Standardized test data indicated significant gains in reading comprehension, 
vocabulary, and total reading scores. Student achievement in mathematics 
included significant gains in computation, concepts, and applications and in 
mathematics as a whole. 

Colton's unique utilization of the outside project evaluator for weekly obser- 
vations and quarterly reviews was outstanding. G. Keith Dolan of California 
State College, San Bernardino, visited the project each week and gathered data 
for program improvement. He presented these data to the project staff in 
writing once each quarter. The project staff replied with quarterly written 
reports, listing the actions they had taken in response to the evaluator's 
recommendations. 
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Program Revisions at Colton 



After the evaluator's first quarterly report in November, 1970, the Colton 
staff listed program revisions in 18 areas, including those identified in the 
paragraphs that follow. 

Additional inservice training time was devoted to individualized mathematics 
instruction, including released time for teachers to visit other schools; consul- 
tations with personnel in offices of county superintendent! of schools and work- 
shops with state college professors; use of SRA mathematics tapes; review of 
new materials; and individual teacher conferences. Instructors exchanged 
students to capitalize on individual teacher strengths. Teachers alternated 
their teaching strategies. The time of three teacher aides involved in the 
project was rescheduled to permit more individual tutoring, including four 
periods per day in the classroom. Classified employees were reevaluated 
to make full use of personnel strengths. Teacher aides were included in 
team planning meetings. Additional volunteer help was secured through the 
use of eighth grade students and student teachers to assist with reading. 
Mathematics goals were rewritten in terms of specific student performance 
and behavioral objectives. The use of parents as tutors outside the classroom 
was studied. The objectives of the counselor were redefined, and priorities 
were assigned to individual counseling and guidance. A test- scoring device 
was purchased to assist the staff with test correcting. General program 
objectives were restated in behavioral terms. 

In addition, the meaning of individualized study trips was clarified. (The 
term "individualized study trips" means that only those students who are 
interested in the subject participate; it does not mean that only one or two 
students participate.) A calendar of study trips was prepared by the instruc- 
tional teams. Guest speakers were scheduled for classroom visits. New 
instructional miniunits in reading were written and prepared for subsequent 
class sessions. Regular weekly staff meetings were scheduled separately 
for reading and mathematics teams. A student advisory committee was estab- 
lished. Efforts were directed toward expanding the group of parent volunteers, 
as recommended by the student advisory committee. Thank you notes and 
Christmas cards were sent in recognition of the efforts of the parent volunteers, 
and congratulatory notea were sent to the parents of students cn the student 
advisory committee. A newsletter was published and sent home. Dr. Dolan 
presented a summary of his quarterly evaluation to the school faculty meeting. 
Inservice sessions were scheduled for regular staff meetings. Reading teachers 
reported on conferences attended. Regular biweekly meeting times and dates 
were established for the student advisory committee. Action was taken to 
remedy the weaknesses of the programs, as identified by the staff evaluation 
questionnaire -- weaknesses in communications, inservice training, use of 
planning time, and articulation with the senior high school. 
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’’Consumer Reports” 



Students reported their liking for the electives^ the after school activities, 
the teachers, the "work- at-your-own- speed" booklets, the miniunits, and 
the eight different periods. They felt that some booklets were too long, some 
classes too short, and some vocabulary words too hard and that there were 
not enough paperbacks in the reading classes. 

Reading teachers reported that the most effective elements of the program 
included the team effort, behavioral objectives (a better way to teach), 
miniunits, weekly .planning meetings, qualified teacher aides, paperbacks 
and magazines, and" the reading booklets. Mathematics teachers felt that 
the most effective elements of the program ipcluded the teacher aides, ' individu- 
alized teaching, the variety of materials, . diagnostic testing and prescriptive 
planning, and the math booklets. 

A principal's advisory group felt that the most effective elements of the 
program were individualized teaching, reading instruction, community aides, 
use of paperbacks, the miniunits, and the outside evaluator. 



Final Evaluation of the Project 

During the project year the project staff directed considerable effort to 
follow through on all the suggestions made by the project evaluator. From 
his point of view, "These dedicated efforts have done much to improve the 
project during the past several months." 

In a final student evaluation of the project, there was a strong positive tone 
to the comments made, with favorable comments outnumbering the unfavorable 
by two to one. More than 64 percent of the students considered the program 
to be helpful, especially in reading. Positive comments referred most often 
to such components as the miniuni-ts, "learning at my own rate," the use of 
paperbacks, and the variety of teachers and classes. 

A final parent questionnaire revealed that the parents were overwhelmingly 
supportive of both the mathematics and reading programs, with nearly 90 percent 
of the respondents indicating that they would like to have their children continue 
in the programs. All but one parent considered the programs to be at least 
"moderately effective." 

Overall, the results of the formal evaluation, student achievement data, 
behavioral objectives, and evaluator's surveys of faculty, students, and parents 
indicated a highly successful demonstration program in Colton for 1970-71. 



§ 6 ^ 

O 

ERIC 



r 







Conclusion to the Colton Project 

In an attempt to answer the question of how to obtain the greatest increase 
in student achievement for the least cost per student, this summary of the 
program and evaluation activities of the Colton project seems to lead to one 
conclusion: Improvement in student achievement appears to be more closely 
related to such factors as individualized instruction, use of local resouvces, 
teacher aides, a variety of teacher-made materials and methods, continuous 
evaluation, and parent-teacher-pupil interaction than it is to such costly items 
as plant remodeling, equipment, hardware, or technical assistance. 
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Appendix C 



Education Code Sections on the Subject 
of Demonstration Programs 



Article 5. Demonstration Programs in Intensive Instmction 

in Reading and Mathematics for Lo'w Achieving Pupils 
(Article 5 added by Stats. 1966 (1st Ex. Sess.), Ch. 106. Effec- 
tive until 91st day after adjournment of 1972 Regular Ses- 
sion) ■ 

Legislah'vB Infant 

6490. It ia the intent and purpose of the Legislature that 
exemplary programs be established for intensive instruction 
in :'eading and mathematics to serve as demonstration projects 
aimed solely at developing, within pupils, above-average com- 
petcc.ee in these basic skill subjects. The program shall be 
developed to serve pupils in grade 7, 8, or 9 who attend school 
in designated areas of disadvantage, and who otherwise would 
find difficulty in achieving complete success in high school. 

It is the further intent of the Legislature that these pro- 
grams in intensive instruction in reading and mathematics be 
operated by school districts directly, or by school districts 
through contract for partial or complete operation with any 
competent public or private agency, foundation or corporation. 
It is also the intent of the Legislature that authority bo 
granted to permit the waiver of any provision of the Education 
Code by the program approving agency, if such is necessary 
for the development of model demonstration programs in the 
intensive instruction in reading and mathematics. 

(Added by Stats. 1966 (1st Ex. Sess.), Ch. 106; amended in 
identical language by Stats. 1969, Ch. 1578 and Ch. 1596. Effec- 
tive September 6, 1969, until 91st day after adjournment of 
1972 Regular Session.) 

Administration and Apportionment of Funds 

6491. Erom the funds appropriated therefor by the Legisla- 
ture to the Department of Education for the purposes of this 
article, the Superintendent of Public Instruction, upon recom- 
mendation of the Director of Compensatory Education, with 
the approval of the State Board of Education, shall admiiii&tcr 
this article and make apportionments to school districts to meet 
the total approved expense of the school districts incurred in 
establishing demonstration programs in the intensive instruc- 
tion in reading and mathematics for pupils in grades 7, 8, or 9. 

(Added by Stats. 1966 (1st Ex. Sess.), Ch. 106; amended in 
identical language by Stats. 1969, Ch. 1578 and Ch. 1596. Effec- 
tive September 6, 1969, until 91st day after adjournment of 
1972 Regular Session.) 



Application for Program by District 

6492, The governiiig board of any district which maintaina 
grades 7, 8, or 9 on account of any school or schools located 
in any area designated by the Director of Compensatory Edu- 
cation pursuant to the provisions of Education Code Section 
6482, may make application to establish and operate a program 
under this article. The application shall be in the form tujd 
shall contain such data and information as the director shall 
specify, 

(Added by Stats. 196JB (1st Ex. Sess.), Ch. 106; amended in 
identical language by Stats. 1969, Ch. 1578 and Ch. 1596. Effec- 
tive September 6, 1969, until 91st day after adjournment of 
1972 Jlegular Session.) 

Waiver of Provisions of this Coda 

6493, The governing board of a school district, in its ap- 
plication, may request waiver of the provisions of any section 
or sections of this code if such waiver is necessary to o.stablish 
and Operate a program under this article. The. need for a 
waiver shall be explained and justified in the application. The 
Superintendent of Public Instruction, upon recommendation 
of the Director of Compensatory Education, with the approval 
of the State Board of Education, may grant, in whole, or in 
part, any such request, 

(ArMed by Stats, 1966 (1st Ex, Sess.), Ch. 106. Effective 
until 91st day after adjournment of 1972 Regular Ses.sion.) 

Operation of Programs 

6-!;9z. The governing board of the school district may make 
application to establish and operate a demonstration program 
under this article directly, or may make application to provide 
for partial or complete operation of such a program through a 
contract with any competent public or private agency, foundOp 
tion or corporation. 

(Added by Stats. 1966 (1st Ex, Sess.), Ch. 106, Effective 
until 91st day after adjournment of 1972 Regular Session.) 

Standards and Criteria by State Board of Education 

6495. The State Board of Education shall adopt regulations 
setting forth the standards and criteria to be used in the evalu- 
ation of applications submitted by school districts. The stand- 
ards and criteria adopted by the State Board of Education, 
among other items, shall include a statement of specific goals 
to 1 •' sought in the program both in terms of pupil achieve- 
ment and for the purpose of establishing a model program, and 
the requirements for evaluation of the program. 

Projects shall be approved only if it can be shown that, if 
successful, the cost effectiveness of the project will he such as 
to be adaptable within the budgets of other similar school dis- 
tricts throughout the state. 
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Projects shall be continually reviewed regarding their effeo- 
tiveness in improving the achievement levels of pupils in read- 
ing and mathematics. Projects which are least cost effective 
shall be terminated and shall be replaced with ones of proven 
effectiveness or by new projects which hold promise of in- 
creased effectiveness. 

(Added by Stats. 1966 (1st Ex. Sess.), Ch. 106; amended in 
identical language by Stats. 1969, Ch. 1578 and Ch. 1596. Effec- 
tive September 6, 1969, until 91st day after adjournment of 
1972 Regular Session.) 

Certification of and Amount of Apportionment 

6496. Upon approval by the State Board of Education of 
an application under this article, the Superintendent of Public 
Instruction shall certify the amount to be apportioned to the 
applicant school district. 

(Added by Stats. 1966 (1st Ex. Sess.), Ch. 106. Effective 
until 91st day after adjournment of 1972 Regular Session.) 

Evaluation Report 

6497. No later than the fifth legislative day of each regular 
session of the Legislature, the Superintendent of Public 
Instruction, upon recommendation of the Director of Com- 
pensatory Education, with approval of the State Board of 
Education, shall submit a report to the Legislature on the 
implementation and evaluation of demonstration programs 
under this article, including the achievement of pupils, an 
analysis of the costs of each project detailed in terms of the 
costs of design, implementation and continuing operational 
expenses, including the degree of cost effectiveness of each 
project. The report shall also include recommendations con- 
cerning improvement, retention, extension or other aspects of 
the program. 

(Added by Stats. 1966 (1st Ex. Sess ), Ch. 106; amended 
in identical language by Stats. 1969, Ch. 1578 and Cli. 1596. 
Effective September 6, 1969, until 91st day after adjournment 
of 1972 Regular Session.) 

Termination of Effect of This Article 

6498. This article shall have no force or effect after the 
91st day following the adjournment of the 1972 Regular Ses- 
sion of the Legislature. 

(Added by Stats. 1966 (1st Ex. Sess.), Ch. 106; amended 
by Stats. 1969, Ch. 1578 and Ch. 1596. Effective September 6, 
1969, until 91st day after adjournment of 1972 Regular Ses- 
sion.) 
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